Governing API Consumption
Governing API Consumption
How can you control how consumers use your API?
Sooner or later every change you introduce to your API will break the integration with at least one consumer. The more generic your API is, the higher the chances of a dependent consumer misbehaving. Why does that happen? And, more importantly, what can you do to prevent it?
This article is brought to you with the help of our supporter: Speakeasy.
Speakeasy provides you with the tools to craft truly developer-friendly integration experiences for your APIs: idiomatic, strongly typed, lightweight & customizable SDKs in 8+ languages, Terraform providers & always-in-sync docs. Increase API user adoption with friction-free integrations.
To begin let's start by introducing the concept of a breaking change. It's critical to understand that the fact something is "breaking" is seen from the perspective of an API consumer.
As I wrote in "Building an API Product," a breaking change is one "that introduces an incompatibility with a previous version" of the API. Examples of breaking changes include removing an operation, adding a new required parameter or payload property, and changing authentication or authorization requirements.
In summary, whenever producers introduce a change that makes a consumer fail to continue to use their API, you have a breaking change. As the number of consumers grows, the chances of a change breaking something approaches one. In the words of Hyrum Wright, "Any changes to [an API] will violate consumer expectations."
Why does that happen? In theory, consumers use your API following a contract expressed by a shared definition. In practice, however, they consume your API in any possible way. Hyrum's law puts it very succinctly:
With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody.
What's interesting here is the sentence "It doesn't matter what you promise in the contract." In other words, your finely crafted API definition doesn't really matter. All the work you put into making sure the design of your API meets the needs of consumers doesn't matter. All the linting rules you meticulously apply to every version of your API definition don't matter. The contract doesn't matter!
The interface stops being the interface. Hyrum goes further and says the "interface has evaporated." I prefer to think of the interface as a lens you can focus on or blur away from:
The more you focus on the interface, the more it becomes visible. Try blurring the interface, and you’ll realize that it’s not that important.
So, if the interface is now gone, how do you control how consumers use your API? And, is it fine to just let things crash whenever you introduce a change? This sounds like another challenge the ruler of API governance has to face. Governing your API by controlling its definition and implementation sounds easier than governing how consumers interact with it.
Another way to look at uncontrolled consumption is to see your API as a dependency every consumer has. The dependency is not exactly your API, really. Instead, it's the parts of your API the consumer chose to use to fulfill a specific use case. Imagine a Web frontend consuming your API in unexpected ways. The API operations that are a part of the consumption become a dependency of the Web frontend. Whenever you make a change to one of those operations there's a probability the Web frontend will break.
I like to call this type of consumption "uncontrolled dependencies." They proliferate in situations where it's easier to not follow a contract and simply make use of what's available at the moment. It becomes natural for consumers to just follow the path of least resistance and use whatever operations they can to implement their feature.
One way to control these dependencies is to align the API with its consumption as closely as possible. By solving use cases instead of offering generic operations you discourage ad-hoc consumption. Integrators prefer to follow ready-to-use operations that adapt to the problems they want to solve, instead of experimenting with a myriad of operations. This approach sounds feasible but in the long term, as the number of use cases and consumers grow, it's unavoidable that you'll get again into uncontrolled consumption.
Another way to establish dominance in the way consumers interact with your API is to completely replace it with something closer to the consumers' implementation. Instead of using your API consumers would import and use code that solves the problems they have. By controlling the end-to-end experience your chances of introducing breaking changes are reduced.
Yes, you'll have to manage how your SDK is put together and you'll have to update it whenever you update your API. However, those are minor drawbacks compared to the challenge of governing widespread uncontrolled API consumption. Especially with internal APIs, this approach has the significant advantage of reducing the friction between consumers and producers.
In the end, the interface becomes the implementation, as Hyrum predicted. The SDK becomes a local embodiment of the API that controls how consumers access it. With an SDK you can enforce usage patterns that you can't with an API definition alone. Yes, consumers could bypass your SDK and use the API directly. However, if you provide a good quality SDK that implements the required use cases and adapts to the consumers' programming language, they won't want to do it.