Designing Web APIs

Chapter 1. What’s an API?

“What’s an API?” When a new programmer asks this question, they typically get the answer, “an application programming interface.”

But APIs are so much more than their name suggests—and to understand and unleash their value, we must focus on the keyword interface.

An API is the interface that a software program presents to other programs, to humans, and, in the case of web APIs, to the world via the internet. An API’s design belies much about the program behind it—business model, product features, the occasional bug. Although APIs are designed to work with other programs, they’re mostly intended to be understood and used by humans writing those other programs.

APIs are the building blocks that allow interoperability for major business platforms on the web. APIs are how identity is created and maintained across cloud software accounts, from your corporate email address to your collaborative design software to the web applications that help you order pizza delivery. APIs are how weather forecast data is shared from a reputable source like the National Weather Service to hundreds of software apps that specialize in its presentation. APIs process your credit cards and enable companies to seamlessly collect your money without worrying about the minutiae of financial technology and its corresponding laws and regulations.

More and more, APIs are a key component of scalable and successful internet companies like Amazon, Stripe, Google, and Facebook. For companies looking to create a business platform that expands the market for everyone, APIs are an important piece of the puzzle.

Designing your first API is just the beginning. This book goes beyond design principles for your first API and dives into how to develop and grow an API alongside your business. With the right choices, your API will stand the test of time.

Why Do We Need APIs?

APIs have emerged out of a need to exchange information with providers of data who are equipped to solve specific problems, so folks at other companies don’t have to spend time solving those problems themselves. For example, you might want to embed an interactive map on a web page without reinventing Google Maps. You might want to have a user sign in without having to reinvent Facebook Login. Or, you might want to create a chatbot that occasionally interacts with users without having to build a real-time messaging system.

In all of these cases, supplementary features and products are created using data or interactions from a specialized platform. APIs enable businesses to develop unique products quickly. Rather than reinventing the wheel, startups are able to differentiate their product offerings while taking advantage of existing technologies and tapping into other ecosystems.

Who Are Our Users?

None of the theory matters if you’re not focused on building the right thing for the right customer.

Bilal Aijazi, CTO at Polly

When building any product, it is a good idea to focus on the customer first. This is also very important when designing an API.  In Chapter 8, we talk about different types of developers and use cases as well as strategies to engage with them and give them value. It is important that you understand who your developers are, what their needs are, and why they are using your API. Focusing on developers prevents you from building APIs that no one wants to use or that do not fit the usage requirements of your developers.

Because changing an API’s design is very difficult after the fact, it is important that you specify your API and validate it long before you start to implement it. The cost of switching from one API design to another is extremely high for most developers.

Here are some examples of developer use cases for an image upload and storage API:

• Lisa is a web developer in a startup that sells art, and she needs an easy way for the artists to upload and show photos of their work.

• Ben is a backend enterprise developer who needs to store receipts coming from the expense system into his audit and policy solution.

• Jane is a frontend developer who wants to integrate real-time customer support chat onto her company’s website.

These are just a few examples, each with unique hidden requirements and needs. If you do not address your developers’ needs, your API will not be successful.

In the next section, we talk about high-level use cases that can have an impact on the design of your API, but the more granular you can be with your use cases and the better you understand your developers, the better you can serve them.

The Business Case for APIs

It’s no secret that the web powers a large portion of new product innovation and the technology market today. As a result, APIs are more important than ever in creating a business, and there are many models for incorporating them into a product. In some cases, an API will lead to direct profit and monetization (via profit share models, subscription fees, or usage fees). But there are also many other reasons you might want to create an API. APIs might support your company’s overall product strategy. They might be a critical piece of the puzzle for integrating third-party services with your company’s product. APIs might be part of a strategy to incentivize others to build supplemental products in which the developer of the main product is unwilling or unable to invest. APIs might also be a way to generate leads, create new distribution channels for a product, or upsell products. For more information on these strategies, see John Musser’s presentation on API business models.

An API must be aligned with the core business, as is the case with many software as a service (SaaS) companies. Notable examples are GitHub, Salesforce, and Stripe. Sometimes, the products built on these APIs are referred to as “service integrations.” Consumer APIs work well if there is a critical mass of user-generated content, such as with Facebook and Flickr’s photo-sharing capabilities. Although there are many reasons to create an API and launch a developer platform, there is also a clear reason not to create a developer platform—when the API strategy is not aligned with the core business. For example, if the product’s main revenue stream is advertisements, APIs that enable alternative “clients” for the product will drive traffic away from the experience where the ads are hosted. That will take away from revenue share, as was the case with the Twitter API.

Monetization and business incentives aside, let’s take a more detailed look at the following ways that some companies have structured their API development:

• APIs for internal developers first, external developers second

• APIs for external developers first, internal developers second

• APIs as the product

APIs for Internal Developers First, External Developers Second

Some companies build their APIs for internal developers first and then release them to external developers. There could be a number of motivations for this. One reason might be that the company sees potential value in adding an external API. This could create a developer ecosystem, drive new demand for the company’s product, or enable other companies to build products that the company itself does not want to build.

To take a look at a specific instance, let’s explore how Slack’s API started—as an API for Slack’s web, native desktop, and mobile clients to display a messaging interface. Although Slack originally created its APIs for its internal developers, something else happened as the company grew: a handful of “integrations” with important business software became a key piece of the puzzle for Slack’s growth and development as communication software. Instead of building bespoke apps to integrate its offering with other companies, Slack launched its Developer Platform and a suite of products for third-party developers to build their own apps, both at established companies and at new startups.

This move on Slack’s part helped to grow the ecosystem for apps that integrate with Slack’s messaging platform. It also meant that users of Slack who also used other business software could seamlessly collaborate where communication was already happening in the Slack messaging client.

The advantage to Slack’s APIs at the time of its Developer Platform launch was that the APIs were already tested and well used by internal developers. The disadvantages to this approach showed up over time as the needs of external developers drifted apart from the needs of internal developers. Internal developers needed flexibility to create new experiences for end users of the messaging client, from new types of shared channels, files, and messages, to other increasingly complex communication experiences. Meanwhile, third-party developers were no longer creating replacement client user interfaces (UIs) for Slack—they started to create powerful business applications and tools that were designed for workflows rather than message display. External developers also required stability, and the tension between API backward compatibility and the need to change the API for new product features had a cost on project velocity within Slack.

APIs for External Developers First, Internal Developers Second

Some companies create APIs for external stakeholders first and then release them to internal stakeholders. That’s how GitHub has operated since the beginning. Let’s take a look at how and why GitHub developed its API and how its developer audience has affected the evolution of the API.

In the beginning, GitHub’s API audience was primarily external developers who wanted to gain programmatic access to their own data. Shortly after the initial release of their API, small businesses began to form around GitHub’s API. These businesses were creating developer tools and selling them to GitHub’s users.

Since then, GitHub has expanded its API offering significantly. It has built an API that serves both individuals who want to create their own personal projects or workflows and teams that want to collaborate to build bot scripts or workflow tools that integrate with GitHub. These teams, called integrators, build developer tools, connect users with GitHub’s platform, and sell these tools to mutual customers.

When it came time for GitHub to build its GraphQL API, third-party developers were the first consumers. GraphQL is a query interface for web APIs. Although it isn’t the first such interface, it gained a bit of buzz prior to the writing of this book due to its implementation by Facebook, a well-known API provider, and its adoption by GitHub, another well-known API provider. After third-party developers began to use GitHub’s new GraphQL API, internal GitHub developers also adopted it to power the GitHub web UI and client applications.

In GitHub’s case, the API had a clear intention to serve external stakeholders first and then eventually evolved to serve internal developers as well. One advantage to this approach is that the API can be customized to serve external developers, rather than straddling two audiences. As GitHub’s API evolved, it was able to annotate its JSON responses with more and more data that developers needed. Eventually, the payloads were so large that GitHub implemented GraphQL so that developers could specify the fields they wanted with the queries. One disadvantage to this approach in the case of GraphQL is that due to the flexibility that GraphQL gives developers, performance bottlenecks that emerge are spread across a variety of access patterns. This can make troubleshooting tricker than when working with a single endpoint at a time, for example in the case of REST.

Note

For more details on GraphQL, see Chapter 2.

APIs as the Product

For some companies, the API is the product. Such is the case with Stripe and Twilio. Stripe provides APIs for payment processing on the internet. Twilio provides APIs for communication via SMS, voice, and messaging. In the case of these two companies, building an API is 100% aligned with a single-product audience. The API is the product, and the entire business aligns behind building a seamless interface for customers. As far as managing APIs and meeting user needs, the API as the product is the most straightforward company arrangement possible. 

What Makes an API Great?

We asked industry experts this question, and the answers we received boiled down to whether the API achieves what it is supposed to do. To delve into the aspects that contribute to an API’s usability, we will not only explore aspects of designing and scaling APIs, but also the support and ecosystems that enable developers to use APIs.

Expert Advice

A good API may come down to the problem you’re trying to solve and how valuable solving it is. You may be willing to use a confusing, inconsistent, poorly documented API if it means you’re getting access to a unique dataset or complex functionality. Otherwise, good APIs tend to offer clarity (of purpose, design, and context), flexibility (ability to be adapted to different use cases), power (completeness of the solution offered), hackability (ability to pick up quickly through iteration and experimentation), and documentation.

Chris Messina, developer experience lead at Uber

Usability, scalability, and performance are some of the aspects that make a good API. We cover many of these topics in Chapters 2 through 4 of this book. Documentation and developer resources are also important to setting users up for success. We cover those in Chapters 7 through 9. Because it is impossible to optimize an API for all factors, the implementation team must make tough decisions about what is most important for the end user. We teach you how to build a strategy to address this in Chapter 7.

One more thing to consider is how a great API will stand the test of time. Change is difficult and inevitable. APIs are flexible platforms that connect businesses, and the rate of change is variable. In large-enterprise contexts, the rate of change is slower than in small startups that have not yet found product–market fit. But sometimes, these small startups provide invaluable services via APIs that enterprises must use. In Chapter 5, you also learn about how to design APIs to stand the test of time and change.

Closing Thoughts

In summary, APIs are an important component of modern tech products, and there are many ways to structure a business using them. In Chapter 2, we give you an overview of API design paradigms.