The Engineering Executive's Primer

Chapter 6. Measuring Engineering Organizations

For the past several years, I’ve run a learning circle with Engineering executives. The most frequent topic is friction with their CEO. The second most frequent is what to measure when their CEO asks for a periodic report on Engineering metrics. Any discussion about measuring Engineering organizations quickly unearths strong opinions. Some of the strongly held perspectives that I’ve heard are:

• Tracking Agile story points is either essential or nearly criminal.

• Objectives and key results (OKRs) are done by every serious company, or rarely done at all.

• Engineering is just developer productivity, which you can track using deploy frequency, cycle time, and so on.

• Tracking incident frequency and impact is very valuable, but tracking incidents is also a harmful antipattern.

• It doesn’t matter what you track, because no one will read it anyway; just make a pretty chart.

• Just measure something related; what really matters is starting to educate your stakeholders rather than the particular metric.

All of these perspectives have a grain of truth to them, but frustrations about measurement are easily overstated. There are many effective modes of engineering measurement, all of which will help you operate as an Engineering executive. You need to measure Engineering to build software more effectively, and to hold your teams accountable for their work. You will also need to measure Engineering to collaborate cross-functionally, and to stay aligned ...