Atoms

I found myself thinking about Spring ‘83 today, an experimental microblogging protocol that had a lot of excellent ideas, but for which the experiment is already over. The protocol’s author wrote up a great retrospective on it.

I recently took the step of banning Reddit from my devices. I enjoy posting longform comments in the few remaining good subreddits, but it’s become a malignant political cesspool that’s unquestionably having a negative impact on the way I think. I’m also increasing unsure of how much content is being produced by humans, as opposed to LLM-powered bots designed to manipulate pubic opinion. I like X, but in the way that I liked alcohol. It’s pleasurable, but cancerous. I’ll continue to avoid Instagram and TikTok for as long as I live.

But I miss seeing thoughtful/high effort posts from interesting people. Spring ‘83 is almost exactly what I’d want out of a prospective social network:

• Everybody gets a single current “slot”, so there’s no incentive to post dozens of times a day.
• No likes/favorites, disincentivizing outrage/sensationalist posting.
• A modest, but significant technical barrier to entry (e.g. generating a key, self-hosting). A little gatekeeping would eliminate the vast majority of the worst people on the internet.
• Non-algorithmic, letting me easily banish anyone who posts about politics.

The realist in me knows that a new high-effort, low-endorphin social protocol catching on is about as likely as me jumping over the moon. As imperfect as RSS and email are, they’re the best channels we currently have, and closest thing to a good answer we’re likely to get. A practical person would focus less on trying to bootstrap a brand new protocol, and more on improving email/RSS to make it more like the protocol we’d want to see in the world.

Published sequence 094, sky blue.

A random shot of the high towers of the Financial District in downtown San Francisco. Used as the poster image for my latest now update, which touches lightly on journaling, OpenTelemetry, and transaction anomalies.

After observing a real, live production transaction anomaly a few weeks ago (more on this soon!), I spent yesterday writing an augmentation for our homegrown Go data loading framework that loads records with SELECT ... FOR UPDATE on endpoints that expect to modify or delete them.

While looking for prior art, I came across ActiveRecord::Locking::Pessimistic module, the Rails equivalent of what I was building:

# SELECT * FROM account WHERE id = 1 FOR UPDATE
Account.lock.find(1)

Use of .lock on update/delete actions will prevent a non-repeatable read, the most common transaction anomaly on Postgres’ default read committed isolation level.

But a lot of people probably don’t know that. Transaction anomalies are rare on the whole, and even when they happen probably aren’t diagnosed and/or fixed. It may me wonder how many shops are consciously practicing good locking hygiene versus how many are YOLOing assuming the side effects are rare and fairly benign. We were certainly the latter.

Published fragment Optimizing JPEGs with MozJPEG for local archival, on writing a wrapper script around MozJPEG to achieve ~80% compression on large JPEGs with little downside.

Blake published an article over on River’s blog on Building an idempotent email API with River unique jobs.

I’ve been using Mailgun to send emails for years, and perhaps the oddest thing about the service is that despite having exactly one critical API endpoint that everyone cares about (the one that sends mail), they won’t give you a way to guarantee that you’re communicating with it safely. Even when you’re one of the consumers who cares enough to bother and willing to do the work themselves, Mailgun won’t give you a way to do it, even after 15 years of product development. I don’t have a good explanation for this. Like I said, odd.

The River post shows that baking in an idempotency concept doesn’t have to be that hard. So if you’re running an API, maybe just do it.

Published fragment The right way to do data fixtures in Go, on a safe, succinct test data fixtures pattern using sqlc and validator.

Published fragment Profiling production for memory overruns + canonical log stats, on using Go’s runtime.MemStats and canonical log lines to isolate huge memory allocations to a specific endpoint.

Published fragment Go’s bytes.Buffer vs. strings.Builder, on taking five minutes to write a benchmark so I know which of these I should be reaching for first.

Published my last fragment of the year, Postgres UUIDv7 + per-backend monotonicity, on how Postgres’ v7 UUIDs are made monotonic, and why that’s a great feature.

Published sequence 093, Eighty-Eight.

I’ve been hearing about Eighty-Eight for years, but didn’t visit until yesterday. Named for the year ‘88 when Calgary hosted the Olympic winter games, it’s two levels of taprooms with the brewery on site right next door. An 80s vibe pervades the space, complete with tube TVs (bunny ears open wide), boombox, and retro console. The special holiday theme is Home Alone. See the infamous Wet Bandit duo on screen here, but also with themed menu, Kevin McCallister traps visible around the room, and flying fox line soaring above the brewery’s enormous stainless steel vats.

Published fragment Stripe V2. How Stripe’s API got a new major version, and no one noticed.

I agree with this: AI-Generated Images Discourage Me From Reading Your Blog.

I have a growing hatred for AI-generated images in blogs. It makes me wonder if the text in the blog posts is AI-generated to some extent. It’s always disappointing seeing these images in blogs run by individuals. I expect this from corporate blogs but not indie blogs.

It might be a losing battle, but I don’t read content that I know is AI generated. It’s harder not to look at an AI-generated image, but I try my best there too.

Last week we ran into a product degradation where a user with an enormous number of tables was seeing timeouts trying to fetch a list of their cluster’s databases from one of our API endpoints. The endpoint had been using pg_database_size() to get sizes for each database, with these being used as a proxy for which were most active to determine the one that should take precedence in the UI.

I verified the problem locally by creating a database with a heck of a lot of tables:

$ createdb many_table_test
$ for i in {1..220000}; do
      psql postgres:///many_table_test -c "CREATE TABLE table_$i (id bigserial PRIMARY KEY)";
  done

And watching pg_database_size become degenerately slow:

# select pg_database_size('many_table_test');
  pg_database_size
------------------
        5180509331
(1 row)

Time: 22739.724 ms (00:22.740)

23 seconds to calculate the size of a database on a fast, local disk.

Taking a look at Postgres’ source, the problem is quickly apparent. Calculating database size involves descending through every one of its files on disk and adding them all up:

/* Return physical size of directory contents, or 0 if dir doesn't exist */
static int64
db_dir_size(const char *path)
{
    int64 dirsize = 0;
    ...

    while ((direntry = ReadDir(dirdesc, path)) != NULL)
    {
        if (stat(filename, &fst) < 0)
            ...
        dirsize += fst.st_size;
    }

    return dirsize;
}

Every table (all 200k+ of them) is a separate file:

$ psql river_test
river_test=# SELECT pg_relation_filepath('table_1234');
 pg_relation_filepath
----------------------
 base/305728/305756

river_test=# SELECT pg_relation_filepath('table_2345');
 pg_relation_filepath
----------------------
 base/305728/305783

I patched the immediate problem by removing uses of pg_database_size and falling back instead to pg_stat_database, which includes a number of statistics that work as rough proxies for size/activity. I used xact_commit, the number of committed transactions. A call to pg_stat_reset() would reset the number, but in any active database it’d grow quickly again.

River Ruby gets same day support for Ruby 3.4. I like Ruby’s tradition of releasing on Christmas morning. It gives me a ten minute job (and ten minutes only) to feel like I did some coding for the day.

Publish fragment Go’s maximum time.Duration, on Avoiding overflows with Go’s time.Duration in the presence of exponential algorithms.