Alex WlchanModify

2026-09-15 19:04:15

I listen to a lot of podcasts, and I like how they fit around other tasks. I press play, lock my phone, and put it down. I’m free to wash the dishes, fold the laundry, or shop for groceries.

Unfortunately, more and more information is only published as a video. Technical talks, conference sessions, video essays – they don’t work in an audio-only podcast app. I could convert these videos to MP3 files, but that breaks down the moment a video isn’t pure spoken word. If a speaker says, “Look at this slide” or holds up a diagram, an audio-only file leaves me stranded.

I don’t want to give up the podcast player I like, nor stare at a screen for an hour – but I do want the information in these videos.

To solve this, I’m abusing my podcast player’s chapter support. This gives me the best of both worlds: I can listen to a video as audio-first, and glance at my lock screen if I need a moment of visual context.

The idea: Chapters every few seconds

MP3 files can have ID3 metadata, and ID3 metadata can include chapters. A chapter covers a particular time range, and it can have an associated title, description, and cover art.

My podcast app of choice is Overcast, which can’t play videos, but it does have robust chapter support. I can jump between chapters, navigate a table of contents, and see per-chapter cover art.

To get videos into Overcast, I’m creating MP3 files with a new chapter every few seconds, and the per-chapter cover art is a corresponding frame from the video. As I play the file, I get a slow, stop-motion-like rendition of the original video. If my phone is locked, I can glance at my lock screen and see the current frame in the Now Playing screen.

Overcast is developed by Marco Arment, and I got this idea from Forecast, his app for adding chapters to podcasts. In particular, I was struck by its ability to create chapters that don’t display in the chapter list – ideal if I don’t want a table of contents with hundreds of entries. As I was developing my script, I compared my output to the output from Forecast to ensure I was creating the chapters correctly.

The code: FFmpeg and Mutagen

There are three steps in this process:

1. Convert a video file to an MP3
2. Extract images from the video at a fixed interval
3. Insert the images as hidden chapters in the MP3 file

Let’s go through each in turn.

1. Convert a video file to an MP3

Converting a video file to an MP3 is a single FFmpeg command:

ffmpeg -i video.mp4 audio.mp3

This is consistently the slowest step of the process, and I do wonder if I could use different settings or an alternative encoder to make it go faster – but it’s not slow enough to be worth further investigation.

2. Extract images from the video at a fixed interval

Extracting images from a video needs a more complicated FFmpeg command:

ffmpeg -i video.mp4 \
  -vf 'fps=1/5,scale=iw*sar:ih,scale=min(iw\,945):min(ih\,945):force_original_aspect_ratio=decrease' \
   thumbnail_%04d.jpg

This extracts an image every 5 seconds, downscales any image larger than 945 pixels square (while preserving the original aspect ratio), and saves the results as sequentially numbered JPEG images (thumbnail_0001.png, thumbnail_0002.png, and so on).

The key is the -vf flag, which defines two FFmpeg filters:

• The fps filter selects one frame every 5 seconds (fps=1/5).
• The first scale filter scales the width based on the sample aspect ratio (scale=iw*sar:ih). Without this filter, frames can be stretched and distorted.
• The second scale filter scales the input video, preserving the original aspect ratio (force_original_aspect_ratio=decrease), and ensuring the output images fit within 945×945px or the size of the input video, whichever is smaller. My limit is 945 pixels because that’s the largest size that cover art is shown on my iPhone.

This filter still isn’t completely correct – it sometimes creates images from portrait videos that are smaller than I’m expecting – but it’s good enough. These are only thumbnails for glancing at, and if I want to change it later, I can always do the image resizing outside FFmpeg.

3. Insert the images as hidden chapters in the MP3 file

Inserting the chapters into the MP3 file is more complicated. Although FFmpeg has basic support for ID3 metadata, as far as I know, it can’t insert chapters with per-chapter artwork. Instead, I’m going to reach for Python and the Mutagen library.

Here’s the code to add a chapter to an MP3 file:

from mutagen.id3 import APIC, CHAP, ID3, PictureType

audio = ID3("audio.mp3")

with open("thumbnail_0001.jpg", "rb") as f:
    img_data = f.read()

image_frame = APIC(mime="image/jpeg", type=PictureType.OTHER, data=img_data)

chapter_frame = CHAP(
    element_id="chp1", start_time=0, end_time=5 * 1000, sub_frames=[image_frame]
)

audio.add(chapter_frame)
audio.save()

This creates a single chapter that lasts the first 5 seconds (0 to 5000 milliseconds), and the per-chapter cover art is thumbnail_0001.jpg. If we ran this in a loop, we could add images for every 5 second slice of the original video.

This code is inserting two frames into the ID3 metadata:

• The CHAP (chapter) frame contains the timing information, and it can have subframes for metadata like title, chapter art, or associated URL.
• The APIC (attached picture) subframe contains information about a picture, which can either be a blob of image data or a URL to an image on the web.

Normally, you’d also insert a CTOC frame which defines a table of contents, but I don’t want a TOC with hundreds of 5-second chapters, so I’m deliberately not doing this here. This is allowed by the ID3 spec – you’re not required to insert a CTOC frame if you’re using chapters, and you can have chapters that aren’t listed in your table of contents.

To work out which frames I needed, I used Forecast to create some chapters by hand, and I inspected their frames. In particular, loading an MP3 and calling Mutagen’s pprint() method shows a human-readable list of frames, and then I could drill into the individual fields:

from mutagen.id3 import ID3

audio = ID3("audio.mp3")
print(audio.pprint())

I wrapped all this code in a project called glancecast, which allows you to convert a video file with a single command, with optional flags to set the frame length and chapter art size:

$ python3 glancecast.py interesting_talk.mp4
interesting_talk.mp3

The process takes a minute or so to complete, most of which is spent transcoding the video file to MP3. The resulting MP3s are usually 40 to 50 MB in size, which is very reasonable.

The outcome: How it looks in practice

Here’s what one of these “glanceable” podcasts looks like in Overcast and on my lock screen:

It’s not a lot of extra information, but enough that I can quickly glance down and get the gist of what a speaker is saying. Both views update with a new frame every few seconds, or I can put my phone in my pocket and ignore the screen.

I’ve used this approach for half a dozen videos so far, and I’m happy with the results. I expect to keep using it, because I have a long queue of videos I’ve been meaning to watch.

If you’d like to try this, check out glancecast for the full code and instructions.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-08-20 17:49:28

I’m working on a project that generates multiple .tar.gz archives, and I need to combine them into one final file. I thought I could just cat the bytes together, but that doesn’t work. This seemingly simple task exposed my flawed understanding of tar and gzip.

To my fix my code, I first had to fix my mental model – and that took me into tape drives, patent laws, and end-of-file markers.

tar stands for tape archive

tar is a file archiver that combines multiple files and their metadata – filenames, timestamps, directory structure – into a single file.

It was originally designed for magnetic tapes, and the file structure is informed by the physical constraints of that medium:

1. Sequential reads. Magnetic tapes are most efficient when you start at the beginning, and play forward to the end of the tape.

2. Append-only writes. Early tapes could only append data to the end of a record, not replace existing data.

3. Fixed data sizes. Tapes have a fixed capacity, and early tapes had fixed data block sizes.

Internally, a tar archive is a sequence of files, each broken into fixed-size blocks. Files have a header block (with metadata like filename and file size) and data blocks (the file contents). After the files, there are two or more blocks filled entirely with zeroes. These form an end-of-file (EOF) marker that tells a reader to disregard everything else in the archive.

This structure mirrors physical tape: you can read files sequentially or append new ones to the end. That sequential design is why tar remains popular for streaming over a network – you can process incoming files immediately, without waiting to download the complete archive.

Knowing this structure helps me understand aspects of tar that I previously found confusing:

• File sizes must be declared upfront. You need to write the file size in the header before you write any data blocks. When I use Python’s TarFile.addfile API, I often forget to set tarinfo.size, so Python writes 0 to the header and creates an empty archive.

• Archives can contain duplicate filenames. You can’t edit or delete existing blocks on tape, so you update a file by appending a new version with the same filename. When you unpack the archive, the later file overwrites the earlier one.

• Everything after the EOF marker is ignored. Because physical tapes have fixed capacities, the EOF marker signals where data ends and empty tape begins. While tools like GNU tar have an --ignore-zeros flag to keep reading past EOF markers, I want to build archives that can be read with the default settings.

I tried a naïve approach of cat-ing tar archives, but that fails because readers stop at the first EOF marker. Instead, I’m combining archives using Python’s tarfile module. I unpack each archive, then copy its members into a new archive which will have a single EOF marker:

import tarfile

def combine_tars(output_file, input_files):
    """
    Combine multiple tar archives into a single archive.
    """
    with tarfile.open(output_file, "w") as out:
        for f in input_files:
            with tarfile.open(f, "r") as src:
                for member in src.getmembers():
                    out.addfile(member, src.extractfile(member))

combine_tars("numbers.tar", ["one.tar", "two.tar", "three.tar"])

This is more code than concatenating raw bytes, but it creates a tar archive that doesn’t need special settings to read.

gzip compresses a single stream of data

gzip is a stream compressor that takes a single file or data stream, and makes it smaller. The compression is lossless, so you can reverse it to retrieve the original file.

Unlike tar, gzip was a response to patent laws, not physical hardware. Reading RFC 1952 which defines the gzip file format, three design constraints reflect the time in which it was created:

1. Patent-free. The gzip tool was written as a free software replacement for compress, a comprssion tool whose underlying LZW algorithm was protected by patents at the time.

2. Streamable. Compressing or decompressing a gzip file must only use a small, bounded amount of memory. In the early 1990s, when RAM was even more scarce and expensive than it is today, the ability to process data in small, continuous chunks was essential.

3. Portable. A gzip file should be independent of the CPU, OS, filesystem, and other aspects of the computer it was created on. We take this sort of portability for granted today, but it wasn’t always a given.

Internally, a gzip file is a sequence of one or more “members”. Each member has a header (with metadata like original filename and modification time), the compressed data, and a trailer (with a CRC32 checksum and uncompressed size). The file ends after the final trailer – gzip doesn’t have EOF markers.

Conceptually, it’s tempting to see members as an analogue for files, but that’s not how gzip works. Tools treat multiple members as part of the same data stream, and you can’t list or extract them individually. When you uncompress a multi-member gzip file, you only get a single stream back.

Because members come one after another and there’s no EOF marker, you can concatenate gzip files by just cat-ing bytes:

echo "one uno eins"    | gzip > one.gz
echo "two duo zwei"    | gzip > two.gz
echo "three tres drei" | gzip > three.gz

cat one.gz two.gz three.gz > numbers.gz

gunzip --uncompress --to-stdout numbers.gz

How do you combine tar.gz archives?

tar and gzip are firm friends. tar combines a directory tree into a single stream; gzip makes that stream smaller. Because they both support sequential reads, .tar.gz is very popular for streaming data over a network – you can start processing individual files before you download the entire archive.

My mistake was trying to combine .tar.gz files using cat. gzip happily combines the compressed members into a single stream, but when tar tries to read the decompressed stream, it finds the first archive’s EOF marker and stops reading.

To combine .tar.gz files safely, I have to extract the underlying members and write them to a new file. That means modifying my Python function above from plain read/write (r/w) to gzip-compressed read/write (r:gz/w:gz):

import tarfile

def combine_tar_gzs(output_file, input_files):
    """
    Combine multiple gzip compressed tar archives into a single archive.
    """
    with tarfile.open(output_file, "w:gz") as out:
        for f in input_files:
            with tarfile.open(f, "r:gz") as src:
                for member in src.getmembers():
                    out.addfile(member, src.extractfile(member))

combine_tar_gzs("numbers.tar.gz", ["one.tar.gz", "two.tar.gz", "three.tar.gz"])

This started as a confusing bug, but it became a fun side quest. Now I understand how these formats work, I understand why my original code doesn’t work, and I understand how I can fix it. I can go back to my project, safe in the knowledge that I haven’t missed a secret shortcut or an obvious optimisation.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-08-14 00:14:06

I wrote a post for the Tailscale blog about a long-running series of corruption incidents, and how they eventually led us to find an SQLite bug that predates my entire programming career. I’m incredibly proud of this, both the work and the blog post.

Before Tailscale, I was coming from smaller teams where I didn’t get to tackle problems of this scale or complexity. This was exactly the sort of tricky, deep technical challenge I wanted to be part of (though I’d rather it hadn’t been quite so stressful)! I’m glad I got to play a small part in these incidents, and I learnt so much from the more experienced engineers I worked with. I never want to hear the words “SQLite corruption” again, but if I do, I’d want to have Tailscalars at my side.

Writing the blog post has a blast, too. The piece transformed from a rough draft into a solid, engaging piece of writing, thanks to thoughtful feedback from many people at Tailscale. Most of my writing is self-edited, and it’s always a pleasure to work with a dedicated editor.

Please check out the blog post if you haven’t read it already – I think it’s a fascinating technical story, and one readers of this site are bound to enjoy.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-07-18 16:00:04

One of my favourite tiny details in this website is my non-breaking spaces. I have code that looks for phrases like “5 cm”, “New York”, or “Objective‑C”, and inserts a non-breaking space/hyphen so they’ll never be split across multiple lines.

This is the sort of typographical nicety that would be handled by a professional typesetter if I was writing a printed book with a fixed layout, but that’s not how websites work. My website is viewed at lots of different sizes, and browsers choose where to insert line breaks. I add these non-breaking characters so browsers know to avoid awkward line breaks.

Previously I was only applying this detail to body text, but today I implemented something similar for <code> elements.

I used a lot of inline code snippets in my last post, and while reviewing it I noticed that several of those snippets had unhelpful line breaks. For example, (?-u:…) was split into (?- and u:…), while the flag --multiline was split with - on one line and -multiline on the other. These line breaks make the post harder to read, with no benefit.

I can understand why they happened – browsers look for characters where they can break lines, and in English that includes hyphens. It’s usually fine to split hyphenated words over multiple lines, but it’s annoying when that happens in code.

I could fix this by replacing the hyphen in my <code> with a non-breaking hyphen, but people copy/paste code snippets and that might change the meaning.

Instead, I wrote a check that looks for <code> elements which are short and contain a line-breaking character, then adds the nowrap CSS class.

import re

def add_nowrap(match: re.Match[str]) -> str:
    """
    Add the `nowrap` class to a `<code>` element if it contains line
    breaking characters.
    """
    contents = match.group("contents")
    if "-" in contents or " " in contents:
        return f"<code class=\"nowrap\">{contents}</code>"
    return match.group(0)

text: str

# Add the `nowrap` class to <code> snippets which are short and
# contain line-breaking characters.
#
# The limit of 15 characters is arbitrary. In longer code snippets,
# wrapping is preferable to avoid leaving excessive whitespace on
# the previous line.
text = re.sub(r"<code>(?P<contents>[^<]{1,15})</code>", add_nowrap, text)

This is paired with a CSS rule that uses the text-wrap property to tell browsers not to wrap across lines:

code.nowrap {
  text-wrap: nowrap;
}

This wasn’t necessary, but I think it makes the site slightly nicer.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-07-18 15:11:26

Recently I’ve been tidying up the subtitles in my local media library. There are two popular file formats for subtitles: SRT (SubRip Subtitle) and WebVTT (Web Video Text Tracks).

I’ve been standardising on WebVTT because it works with the HTML5 <video> element, and I play all my videos through the <video> element embedded in static websites. However, lots of subtitles are only available as SRT, so I wrote a Python function to convert SRT files to WebVTT. The formats looked simple and the conversion seemed straightforward. Famous last words!

When I spot checked the converted subtitles, I noticed a bug in my handling of byte order marks (BOM), and it took several steps to fix.

Failing to look for the UTF‑8 BOM

A byte order mark is a special use of the zero width no-break space character U+FEFF at the beginning of a text file, which tella a program reading the file about how the text is encoded. It depends on the exact sequence of bytes used to encode the character. Here are a few examples:

• EF BB BF – the file is UTF‑8 text. UTF‑8 always has the same byte order, so it’s just telling us about the encoding.
• FE FF – the file is UTF‑16 text, with big-endian byte order (UTF‑16BE).
• FF FE – the file is UTF‑16 text, with little-endian byte order (UTF‑16LE).
• 00 00 FE FF – the file is UTF‑32 text, with big-endian byte order.

All of my SRT input files were UTF‑8 encoded, and some of them had the UTF‑8 byte order mark, and I wasn’t handling it correctly. For example, suppose I had this input SRT file:

<U+FEFF>1
00:00:01,001 --> 00:00:10,010
You have grown, Keyne.

2
00:02:00,002 --> 00:20:00,020
Soon you’ll be needing another name.

When I convert to WebVTT, I want to add the WEBVTT header, remove the sequence numbers, and change the timestamp format.

To remove sequence numbers, I was checking if a line was all digits. Because the BOM is on the same line as the first sequence number, the line isn’t all digits, so I didn’t remove it. Instead, I copied the entire line into the middle of the WebVTT file, BOM and all:

WEBVTT

<U+FEFF>1
00:00:01.001 --> 00:00:10.010
You have grown, Keyne.

00:02:00.002 --> 00:20:00.020
Soon you’ll be needing another name.

The correct conversion would remove both the byte order mark and that first sequence number:

WEBVTT

00:00:01.001 --> 00:00:10.010
You have grown, Keyne.

00:02:00.002 --> 00:20:00.020
Soon you’ll be needing another name.

In my local media library, I can assume everything is UTF‑8. I can safely remove the byte order marks, and my web browser will still decode my subtitles correctly.

Fixing the converter with encoding="utf-8-sig"

In my first fix, I tried to handle the BOM manually. I wrote code that looked for U+FEFF and stripped it from the file, trying to detect it and re-insert it into the converted WebVTT file. (This was before I realised I could just remove it entirely.) It was a bit messy, because I was mixing low-level text encoding code with my high-level subtitle conversion steps.

As I was researching this article, I realised there’s a more elegant solution: if I open the SRT file with encoding="utf-8-sig", Python will automatically detect and skip the optional UTF‑8 encoded BOM at the start of the file. The rest of my code doesn’t know or care that it’s there.

I fixed the bug in my function, which means future conversions will work correctly – but what about the broken files I’ve already generated?

Detecting the UTF‑8 BOM with ripgrep

Initially I tried searching for U+FEFF with TextMate, but it crashed consistently with that search, so I turned to command-line tools.

I use ripgrep for searching text. By default it does “BOM sniffing” on files – when it reads a file, it looks at the first few bytes, transcodes the file from its actual encoding to UTF‑8, then executes the search on the transcoded version. This is exactly how the BOM is meant to be used, but it’s less helpful if the BOM itself is what you’re searching for!

Instead, we can disable ripgrep’s Unicode support and search raw bytes by using (?-u:…) in the regular expression. (This flag comes from Rust’s regex crate.) The following command looks for lines that start with the UTF‑8 BOM:

$ rg '^(?-u:\xEF\xBB\xBF)'

If you were only looking for the BOM at the start of the file, you’d also want the --multiline flag. That changes the caret ^ to anchor to the start of the file, not the start of any line. But since I’m looking for BOMs which are in the middle of the file, omitting --multiline is correct.

This search threw up dozens of files with a broken BOM. Initially I opened the broken files in TextMate and edited them manually:

$ rg --files-with-matches --null '^(?-u:\xEF\xBB\xBF)' | xargs -0 mate

But I quickly realised this was too slow, so I wrote a Python script to clean up all the files at once:

#!/usr/bin/env python3

import glob

for filepath in glob.glob("**/*.vtt", recursive=True):
    with open(filepath, "rb") as f:
        content = f.read()
        
    if b"\xef\xbb\xbf" in content:
        content = content.replace(b"\xef\xbb\xbf", b"")
        with open(filepath, "wb") as f:
            f.write(content)
        print(filepath)

Once I’d run this script, I used my ripgrep command to check it was correct – and indeed, all the erronous BOMs had been stripped from my media collection. I also track my subtitle files in a Git repo, so I could confirm the script didn’t introduce other changes.

Before this bug, I’d only vaguely heard of byte order marks, and I’d never had to tackle them in anger. This sort of lesson is exactly why I love managing my local media archives as hand-built static websites – the lo-fi approach gives me lots of opportunities to explore low-level ideas and learn how things actually work on my computer.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-07-14 04:44:05

At work, we use a standard Git workflow: develop on a feature branch, push to GitHub, and open a pull request to main. Once somebody else approves the PR, the changes get merged.

At least once a week, I forget to branch and commit changes directly to my local main.

I only realise my mistake when I try to push and GitHub blocks me. To untangle myself, I have to create a new branch with my current state, push that instead, and then reset my local main back to origin so I can pull other people’s changes. This isn’t difficult to fix, but it’s annoying – especially when I often forget to clean up my local main until the next time I try to pull.

To stop me getting into this state, I’ve written a Git pre-commit hook. I saved the following shell script in .git/hooks/pre-commit and made it executable:

#!/usr/bin/env bash

set -o errexit
set -o nounset

branch="$(git rev-parse --abbrev-ref HEAD)"

if [ "$branch" = "main" ]; then
   echo "You can't commit directly to main"
   exit 1
fi

The git rev-parse command prints the short name of the current HEAD. If I’m on a branch, it returns the branch name; if I’m in a detached HEAD state, it returns HEAD.

If the hook detects that I’m on main, it exits with an error code. This aborts the commit and prevents it being saved, serving as a friendly reminder to create a feature branch first – and leaving my local main completely clean.

Ideally I’d always remember to branch when I start a new piece of work – but since I don’t, I’m happy to let the computer remember instead.

[If the formatting of this post looks odd in your feed reader, visit the original article]