An old problem with APIs of all kinds is that they’re difficult to change. Although an API can be expanded without too much trouble, it becomes costly to remove any fields that may have been present previously in case some consumer has come to be dependent on them. As companies like Twitter found out, breaking API consumers on a consistent basis is the fastest way to foster a disparaging development community and a long-lasting infamy as an unreliable provider.
Modern web pundits might tell you to solve this problem with Hypermedia, but although this type of RESTful technique might provide some protection against the relocation of resources, it can do little to protect against the removal of fields on a resource or the removal of entire resource types.
Many providers try to give themselves a bit of freedom in this area by introducing versioning into their APIs so that in the case of required API changes the version can be bumped and old consumers can keep working. For example, we require calls to our modern API version to specify a version on every HTTP request via the Accept header:
Accept: application/vnd.heroku+json; version=3
But when it comes to web APIs, versioning is problematic in its own way. Anytime that a version is incremented, an orphan is left behind. That orphan still has many consumers left on it, and will require considerable product and technical work in the form of a sophisticated deprecation schedule and strategy. To ensure that consumers aren’t constantly chasing a moving target, that schedule may have to stay very conservative by allowing a grace period of months, or even years. For example, our compatibility policy states that production resources will remain available for twelve months after deprecation. Especially when it comes to building prototypes and experiments, this kind of expense associated with any kind of obsolescence is a hard pill to swallow.
To help improve on this situation, we introduced a concept that we’ve been using for a few months now called version variants. Variants are a simple way of hiding new API features behind a flag so that they stay out of the main API version. They have names that mirror their associated version that look like version=3.new-feature and are requested in a similar fashion:
Accept: application/vnd.heroku+json; version=3.new-feature
Variants have a few important characteristics:
At their core, variants are a tool to ease the prototyping process by making the process of deprecating a prototype cheaper. Their common lifecycle looks a little like the following:
We’ll normally have an API engineer run a full review on the new APIs at the time of pulling a variant mainline, with only minimal manual guidance provided up to that point (we do of course encourage everyone to read our general HTTP API design guidelines before starting anything at all, and to come to us with any proposed designs that don’t fit our existing patterns well). This helps to cheapen the cost of the prototype in that a team building a new feature doesn’t have to swallow the process of an API audit with every change that they make to it.
Our API responds with mainline even for API variants that it doesn’t know about (i.e. 3.*). This makes the process of pulling variants to mainline safe in that even consumers that are still requesting the old variant have their requests filled appropriately until they can be updated.
In the latter case of a prototype’s complete removal, some consumers may be broken just like if a major feature was removed from the mainline API, but hopefully the number of broken consumers will be fewer and the lowered stability expectations of those users will help them cope with the change. In any case, we’d still recommend announcing the deprecation at least a few weeks in advance to provide consumers with some grace time to help them react appropriately.
One anti-pattern that might manifest without careful consideration are prototypes in variants that are not made either generally available or deprecated appropriately, a common case for any project which is started but then loses steam and isn’t finished. To help mitigate this, we’re experimenting with requiring all variants to be assigned an expiry date, after which a variant may be removed liberally if the team that created it is no longer taking appropriate action to continue moving its lifecycle forward. This is modeled in part on the IETF’s guidelines for Internet drafts which require that an expiration date of 185 days from the date of submission is added to the first and last pages of any draft document.
Did I make a mistake? Please consider sending a pull request.