Werner Vogel’s 6 Rules for Good API Design – InApps Technology 2022

If there was any one company that should know how to build Application Programming Interfaces (APIs), it should be Amazon Web Services. The cloud giant processes 1/2 billion API calls a second just for its Identity and Access Management services alone — which log users and machines into AWS cloud resources.

Still, the company is still learning how to best create and manage APIs, as AWS Chief Technology Officer Werner Vogels, pointed out in his keynote at the AWS re:Invent user conference, being held this week in Las Vegas.

He described an early misstep from the company in terms of API design. Each AWS service is managed by a team, and each team is focused on “addressing exactly the needs of our customers and work backwards from what they need,” Vogels told the audience. Citing the dangers of premature optimization, Vogels warned it would be very risky to think about designing a uniform format for APIs before understanding how they will be used would “stifle innovation.”

But the problem with each team focusing on customer needs first means there would be no central coordination of how APIs are designed. As a result, each team’s APIs can differ slightly as to their API formats. For instance, the API call for a Lambda function is structured differently than a call for Kinesis:

aws lambda get-function --function-name WernersFunction

aws kinesis describe-stream --stream-name WernersStream

Taken as a whole, these two are not consistent in how they call an API. This leads to a loss in the productivity of developers, who have more API structures to memorize, and more work to do scripting these API calls. This lack of consistency has been a sore point with customers and partners, Vogels admitted.

So, to solve this problem, the company released the Cloud Control API in September. Cloud Control allows users to create, read, update, delete, and list (CRUD-L) cloud resources using standardized set of application programming interfaces (APIs). For the user, this eliminates the need to generate code or scripts specific to each individual service responsible for those resources. For API designers, it provides a way to continue to support older API versions, while also keeping those services available in this other, more standardized format.

Acting as a unified control plane, Cloud Control works not only for AWS resources, but third-party ones as well. HashiCorp for instance, released a Cloud Control connector to automate Terraform-based deployments on AWS.

Six Best Practices for API Design (and One Tool)

Vogels pointed out that this year marks the 15th year that AWS has been in business, and with that in mind, he listed a few of the lessons learned AWS has picked up over time creating customer-facing APIs. Here are six:

Once an API is created, it should never be deleted, or changed. “Once you put an API out there, businesses will build on top of it,” Vogels said, adding that changing the API will basically break their businesses.

• Never Break Backward Compatibility

Backward capability is a must. This is not to say you can’t modify, or improve the API. But whatever changes you make shouldn’t alter the API such that calls coming in from the previous versions won’t be affected. As an example, AWS has enhanced its Simple Storage Service (S3) in multiple ways since its launch in 2006, but the first-generation APIs are still supported.

• Work Backwards from Customer Use Cases

The way to design the APIs is to not start with what the engineers think would make for a good API. Instead, figure out what your users need from the API first, and then “work backwards from their use cases. And then come up with a minimal and simplest form of API that you can actually offer,” Vogels said.

As an example, Vogels described an advertisement system that can be used for multiple campaigns. Instead of modifying the API for each new campaign, by embellishing it with multiple flags, maintain a core campaign API call, and then build other, compatible, APIs around it. Keep your API’s as simple as possible, so they can be building blocks for others to use.

• Create APIs That are Self Describing and Have a Clear, Specific Purpose

Documentation, of course, should always be up to thorough and up-to-date, but the APIs themselves should also be intuitive upon first glance. “You should be able to look at the API, and always be able to figure out what it does,” Vogels said.

• Create APIs with Explicit and Well-Documented Failure Modes

Speaking of documentation, describing the API’s failure modes — faulty input parameters, security considerations — need to be included as well. “Because it’s not only about what your API does, it is how it fails,” Vogels said.

• Avoid Leaking Implementation Details at All Costs

A leaky implementation is one where the underlying implementation details become part of the API itself — a reference to a specific piece of software, for instance. This can be problematic as the customers will grow to rely on these non-essential details, and “you can no longer change that implementation,” Vogels said.

One AWS tool that can be of help here is Smithy, an open source program that helps organizations collaborate on API design. It generates API templates that can be easily covered in other formats, such as OpenAPI.