Chapter 19. Developing On-Call Resilience
The most visible responsibility of supporting a service or system is on-call and managing impactful events. When your alerting system constantly pages you, you may not have the time or energy to improve the system’s infrastructure effectively. In extreme situations, you may avoid thinking about the on-call experience when you are not on-call because it feels better to accomplish project work. In this chapter, I propose a framework for building resilience, investing early and regularly to prepare for on-call so that you can cope with the challenges and stress that come from being available to handle any issues that arise.
What Is On-Call?
On-call is a temporary rotating role assignment that may include being reachable outside of normal business hours (e.g., evenings, weekends, and holidays) to answer requests for support and handle discovered alerts. When you are on-call, you are one of the people responsible for this work for a specific length of time. Depending on the size and distribution of the team, on-call rotations may consist of 8- to 24-hour shifts for one to two weeks.
On-call duties vary widely within different organizations, from failed application services to power outages. You may be the person to respond to services going offline or provide escalation support. You may have to investigate why a website went offline in the middle of the night or scramble to restore backups when a file server crashes. Some on-call is for the ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access