Atoms

DPReview.com is shutting down on April 10th.

G’damn. This is one of maybe three to five websites that I visit every day. It’s well known for its in-depth reviews, but was also one of the few sites out there that hadn’t degenerated into the labyrinth of clickbait and dark patterns that is the modern web. If I wanted to see which lenses Canon’s released this year or last, DPreview is the only place I’d think about getting that information, far more usable than even Canon’s official site.

I’d forgotten that Amazon had bought them – a shame. An underperforming division of the trillion-dollar everything-store company might be a reasonably sustainable independent business on its own terms.

Published fragment on last week’s layoffs from Meta, on Mark Zuckerberg’s words on manager-heavy organizations and remote work.

The crypto crowd is at it again, with the latest being Balaji saying he’d bet $1M that Bitcoin will be worth $1M within 90 days (current price is $26k). Multiple high net worth individuals have committed to taking the bet. I assume what happens next is that the idea is memoryholed, but this could be an epic win for crypto-skeptics if it takes.

Published fragment Policy on util packages.

Stated simply, general util packages aren’t allowed because they tend to become dumping grounds. But, specifically targeted util packages with narrow, focused domains, are.

util/
    assetutil/
    cookieutil/
    cryptoutil/
    emailutil/
    jsonutil/
    maputil/
    passwordutil/
    ptrutil/
    randutil/
    signingutil/
    sliceutil/
    stringutil/
    testutil/
    timeutil/
    uuidutil/

Published fragment CL.THROTTLE implemented in DragonflyDB, in which the rate limiting command from redis-cell is implemented in DragonflyDB.

CL.THROTTLE user123 15 30 60 1
               ▲     ▲  ▲  ▲ ▲
               |     |  |  | └───── apply 1 token (default if omitted)
               |     |  └──┴─────── 30 tokens / 60 seconds
               |     └───────────── 15 max_burst
               └─────────────────── key "user123"

Published fragment RFCs and review councils, prompted by Squarespace’s write up on their RFC and review process, which sounds very close to Stripe’s.

Published sequence 043, outpost.

Downskill skiing is a very uneven sport. Trips are booked months in advance, and the time you get there, you’re at the mercy of the weather gods. Occasionally you can be buried in snow, like the Tahoe region was this last week with 140”+ of fresh powder. Other times you get nothing, and have to find ways to make your own fun under scratchy conditions.

After years of bad luck skiing BC’s interior, we finally got a good one. A couple big dumps last week before we arrived to establish a base, and fresh snow Sunday, Monday, Tuesday, and Thursday while we were here. What a year.

A small CI improvement we made last week: when running Go workflows we’d generally define a variable for GO_VERSION near the top and pass it into each actions/setup-go below:

env:
  GO_VERSION: 1.20

steps:
  - name: Install Go
    uses: actions/setup-go@v3
    with:
      cache: true
      check-latest: true
      go-version: ${{ env.GO_VERSION }}

GitHub Actions don’t allow steps to be extract and moduralized in any way, so with many jobs in the workflow, the actions/setup-go step needs to be duplicated many times over. Defining GO_VERSION meant that when upgrading to a new version of Go, we only had to change one value in the file.

But it still wasn’t optimal because when upgrading to a new version of Go, we’d have to change the version in go.mod (twice actually, because we’re on Heroku and Heroku’s dumb +heroku goVersion 1.20 magic comment is mandatory), and then also in every GitHub Actions workflow .yaml file. It’s easy to forget one and break something.

There’s an easy solution. It turns out that actions/setup-go supports a go-version-file directive that reads a version out of a go.mod file:

steps:
  - name: Install Go
    uses: actions/setup-go@v3
    with:
      cache: true
      check-latest: true
      go-version-file: "go.mod"

Now when we upgrade to a new Go, only one file is changed.

From Amazon Route 53: Workload isolation using shuffle sharding.

In short, Route 53 domains are sharded by so they map to one of 2,048 virtual servers. The sharding process means that some domains share servers with other domains, and under a normal sharding algorithm, domains would share all their servers with those same domains on their shard. This isn’t great because if I’m an unlucky customer on the same shard as github.com who’s getting DDOSed all the time, if their shards are taken down that means I’m down too. “Shuffle sharding” defines sharding such that even if I share servers with github.com, some of the servers I map to are “shuffled” so that I’m also guaranteed to not share some with github.com. Even if every server hosting github.com goes down, some of mine are still up.

The link above explains it in more detail and with diagrams.

Last week, in a change presumably driven by LastPass’ recent compromise, Bitwarden released support for Argon2. We’d moved to Argon2id just two days before. Some good news about LastPass’ loss is that it’s been everybody else’s gain as other products take notice and shore up their own security.

After a three week odyssey involving cascading dependency failures and 24-hour build loops, Go 1.20 is released on Homebrew.

Published fragment Findings from six months of running govulncheck in CI.

Vulnerability #1: GO-2023-1571
  A maliciously crafted HTTP/2 stream could cause excessive CPU
  consumption in the HPACK decoder, sufficient to cause a denial
  of service from a small number of small requests.

  More info: https://pkg.go.dev/vuln/GO-2023-1571

  Module: golang.org/x/net
    Found in: golang.org/x/net@v0.6.0
    Fixed in: golang.org/x/net@v0.7.0

    Call stacks in your code:
Error: client/awsclient/aws_client.go:156:34: awsclient.Client.S3_GetObject
    calls github.com/aws/aws-sdk-go-v2/service/s3.Client.GetObject,
    which eventually calls golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip

A follow up from yesterday’s post on GitHub being close to having ended up as a Postgres stack, somebody pulled Tobi from Shopify into the conversation, and it turns out Shopify was a near miss on Postgres too:

Yea. Shopify beta was on Postgres. But in our case it was the poor state of replication (in 2005) that made me switch to mysql before launch.

Published fragment Honest health checks that hit the database.

errGroup, ctx := errgroup.WithContext(ctx)

errGroup.Go(func() error { return checkDatabase(ctx, svc.Begin) })
errGroup.Go(func() error { return checkDatabase(ctx, svc.BeginEphemeral) })

if err := errGroup.Wait(); err != nil {
    return nil, apierror.NewServiceUnavailableErrorf(ctx, "Health check error: %v.", err)
}