Architecting for Scale, 2nd Edition

Chapter 5. Dealing with Service Failures

One of the vulnerabilities in building a large microservice-based application is dealing with service failures. The more services you have, the greater the likelihood of a service failing, and the larger the number of other services that are dependent on the failed service. How can you deal with these service failures without adding instability to your application? In this chapter, we will discuss some techniques to deal with service failures.

Cascading Service Failures

Consider a service that you own. It has several dependencies, and several services depend on it. Figure 5-1 illustrates the service “Our Service” with multiple dependencies (Service A, Service B, and Service C) and several services that depend on it (Consumer 1 and Consumer 2). Our service is dependent on three services, and our service is depended on by two services.

Figure 5-1. Our Service and its dependencies and consumers

What happens if one of our dependencies fails? Figure 5-2 shows Service A failing.

Unless you are careful, Service A failing can cause “Our Service” to also fail, since it has a dependency on Service A.

Figure 5-2. Our Service with a failed dependency

Now if “Our Service” fails, this failure can cause Consumer 1 and Consumer 2 to fail. The error can ...