API Traffic Management 101

Chapter 5. Scaling Traffic

One of the biggest challenges to successfully managing API traffic is dealing with the network on which the API travels. Monolith-style implementations need to deal with the network because every API call starts and ends with a network call. Microservice-style implementations, however, are usually more vulnerable to network failures because there are usually multiple network “hops” involved in completing just one external API call. In fact, this is one of the big drawbacks of adopting a microservice approach—the increased dependency on the network to complete your API calls.

In this chapter, we discuss the importance of dealing with network-related problems and review a handy set of patterns you can use to meet most of the challenges of unreliable networks. It is important that your API traffic platform is able to recognize and mitigate these common network failures in order to keep your system up and running at a level of reliability and resilience you and your API consumers expect.

And the first step in this journey to reliability is to recognize that instead of focusing only on preventing network errors, you should also focus on surviving them when they happen.

Surviving Network Errors

Many of the network problems you’ll encounter are things that happen outside your own internal network and events that are out of your control. Your service provider can experience upstream outages, Domain Name System (DNS) routers can be misconfigured, other services ...