Seeking SRE

Chapter 31. Elegy for Complex Systems

Mikey Dickerson, Layer Aleph (formerly United States Digital Service)

March 19, 2018: Farmhouse Motel, Paso Robles, California. Diverted here and lost a day on the way south, because of a rainstorm that was too much for a motorcycle. 18 hours left to make it 220 miles to the next appointment. This isn’t going to plan. Nothing ever goes to plan.

Six years ago, I was a middle manager in SRE at Google. I was about three years into an enormous, and mostly unnecessary, project to reinvent MySQL on Google storage, and my last two promotion applications had not been read by anyone. I was, in the phrasing that our HR person liked to use, “loose in the saddle.” The high-energy collision that sent me completely out of orbit was that I was asked by President Obama to attempt a last-ditch rescue of healthcare.gov, the so-called “federally facilitated marketplace” that was threatening to sink the Affordable Care Act, and with it, the concept of universal healthcare for another generation.

The healthcare.gov rescue grew into a round-the-clock effort by a couple dozen SREs, mostly from the Bay Area. Most of them today agree they would not trade the experience for anything, and also that they would never wish a similar experience on anyone else. The rescue in turn grew into a new government entity, the US Digital Service. And we were set loose on the most stubborn IT issues in the US government, most of which started out as an attempt to replace one large ...