Building Serverless Applications on Knative

Chapter 9. Failing at the Speed of Light

So far I’ve been a cheerleader for serverless technologies and systems, but this chapter will be an opportunity to explore the things that can go bump in the night when deploying and living with serverless at scale. We’ve talked a lot about how serverless systems built on relatively new technology scale easily and reduce the maintenance costs for developers and system administrators. The exciting thing about new technologies is that they can completely erase problems that are intractable in existing architectures. New technologies also bring new failure modes—every revolution has some winners and losers. Still, evaluating the trade-offs, many of us (including myself) will prefer serverless systems when they are available—but we’ll also remember where the sharp edges are and design for them the same way we designed for the sharp edges of traditional systems.

Many of the failure modes we’ll talk about in this chapter are related to the scaling capabilities of serverless runtimes. Sometimes these are problems that traditional systems also suffer from, but that fail differently when rapid scaling is a possible solution to a capacity problem. Others are side effects that occur because infrastructure realities don’t match up with our theoretical models¹ or because not all infrastructure is equally serverless. In many of these cases, the solution to the problem is simply to enforce some upper limits on the system scaling, but understanding the ...