Databases shed important RDMS features as they get big. Examining why this tends to be the case, and some ideas for preventing it.
Making the sorting speed of network types in Postgres twice as fast by designing SortSupport abbreviated keys compatible with their existing sort semantics.
In the spirit of the continued production of independent content, I’m running a two-week photography and writing project in Berlin called Sequences.
A walkthrough of the design of a live reload feature for the static site generator that builds this site, touching on fsnotify, WebSockets, and the curious case of file 4913.
How Postgres makes sorting really fast by comparing small, memory-friendly abbreviated keys as proxies for arbitrarily large values on the heap.
Hitting the limit for maximum allowed connections is a common operational problem in Postgres. Here we look at a few techniques for managing connections and making efficient use of those that are available.
How a seemingly minor enhancement in Postgres 11 fills one of the system’s biggest operational holes.
Using the design principles of a 10,000 year clock to build a program that’s intended to run on a macro timescale.
Why it makes sense to model APIs as graphs, and what GraphQL can do for us to help with discovery, batch operations, and gradual enhancement.
Flattening a single very hot vertical Redis node into a horizontal Redis Cluster at Stripe.
A detailed look at the frameworks, concurrency model, error handling, middleware constructs, and testing strategies of a web service written in Rust.
Why Go’s stability and simple deployments is a good fit for a serverless environment.
Scaling out operation with read replicas and avoiding the downside of stale reads by observing replication progress.
Building a log-based architecture that’s fast, efficient, and resilient on the new stream data structure in Redis.
Building resilient services by identifying foreign state mutations and grouping local changes into restartable atomic phases so that every request can be driven to completion.
When it comes to streaming APIs, there’s now a lot of great options like SSE, GraphQL subscriptions, and GRPC streams. Let’s examine whether webhooks are still a good choice in 2017.
Building a robust background worker system that leverages transactional isolation to never work a job too early, and transactional durability to never let one drop.
Part one of a series on getting easy data correctness by building APIs on the transactional machinery of Postgres.
Why Ruby’s scheme for memory allocation doesn’t play nicely with copy-on-write, and how a compacting garbage collector will help.
A dive into the mechanics that allow Postgres to provide strong atomic guarantees despite the chaotic entropy of production.
Building a toolchain for sending a newsletter, and the dismal state of HTML and CSS in email.
Lets talk about the bad aspects of the technology we use as well as the good. Better information will lead to better decisions in the next generation of technical architecture.
A non-exhaustive primer of modern cloud database solutions like Aurora, Cosmos, and Spanner.
On ensuring system integrity, operability, and correctness through a solid foundational database, and how ACID transactions and strong constraints work in your favor. Why to prefer Postgres over MongoDB.
Practicing minimalism with the lofty goal of total ephemeralization to build coherent, stable, and operable stacks.
Musings on the next API technology, and whether REST-ish JSON over HTTP is just “good enough” to never be displaced in a significant way.
Given a wide API surface area are small changes, most API upgrades are safe for most users. Could we upgrade their fixed API version automatically?
How we overvalue the wrong technology and novel aspects of interface design at the expense of substantial gains to our productivity.
A lightweight and technology agnostic operational technique for easy and near realtime visibility and live querying into production systems.
Why and how to put limits and rate limits on your web services. Introducing redis-cell.
Error handling and fragility; or why a worker pool belongs in Go’s standard library.
The basics of Postgres development: how to build the project, run the test suite, and format a patch for submission.
The case for a concerted effort to build a powerful, but streamlined, platformon AWS.
Building a static site on AWS with a global CDN, free HTTPS with automatic certificate renewal, and a CI-based deployment process powered by GitHub pull requests.
Notes on the language after spending a few weeks building a large project in it.
Getting a certificate that most browsers would accept used to be difficult and expensive, but that’s changing as we enter a relative golden age of web encryption. Read this for options for getting certificates issued for free.
Some of my favorite practices and ideas from almost four years at Heroku.
Using backslash commands in psql to navigate and describe object hierarchy in Postgres and Redshift.
Crunching running data with prepared statements in Postgres.
Implementing rate limiting using Generic Cell Rate Algorithm (GCRA), a sliding window algorithm without a drip process.
A set of general guidelines to consider when designing a alerts for a production system.
How Postgres’ concurrency model coupled with long-lived transactions can degrade the performance of indexes on hot tables in your database.
A short write-up on findings, limitations, and opinion on Kinesis after a month in production.
Splitting and merging in action.
On guaranteeing order with the bulk put API of an event stream.
A simple mechanism for managing changes to a web API and to help cheapen the disposal of prototypes.
A few words on the Unix philosophy of building small programs that do one thing well, and compose for comprehensive functionality.
Learn how to start encrypting dotfile secrets with GPG, and some techniques for getting those encrypted files integrated with your toolchain.
Understanding the benefits of Golang’s restrictive (but simple) import and package management system.
An exercise of discovery around how to extend the shell’s API.
Data warehouses aren’t just for the enterprise! Let’s examine a few of their basic characteristics, and build some examples with Go/Ruby and Postgres.
If you find that the price tag for a Fuji-official adapter ring for the X100S is a little hard to swallow, this 5-minute hack will save you 90%.
A quick tour through some of the fundamentals of JSON Schema and Hyper-schema, and how they can be applied to building APIs.
Stubbing distributed services with Sinatra-based services stubs and enabling stronger constraints with JSON Schema.
Useful distinction or new buzzword? Comments on 200-500 line services.
Interactors by a different name.
When the use of an anti-pattern is considered beneficial.
When building an app against a web API, do you pull in their SDK or just make raw HTTP calls? Here are a few reasons that I avoid SDKs when I can.
In search of an alternative to BreakTime. The discovery of a very classical solution.
How the page almost transitioned successfully to the digital world, but is in decline in new media. The lessons that we can learn from this age-old design element, and why we should hope for its re-emergence.
A logging format used inside companies such as Heroku and Stripe which is optimal for easy development, consistency, and good legibility for humans and computers.
A set of patterns to make APIs more accessible to developers; lowering the barrier of entry for new users, and easing the maintenance of consuming applications.
Designing scope for Heroku OAuth, and a brief tour of other implementations on the web.
How we build minimal, platform deployable, Rack service stubs to take the pain out of developing applications that depend on an extensive service-oriented architecture.
A simple pattern for tracing requests across a service-oriented architecture by injecting a UUID into the events that they produce.
Most of my older work can be found on Mutelight.