5 automation trends in software development, quantified

Lessons from hundreds of development practice assessments across the industry.

By Joost Visser
August 23, 2018
5 movie slide 5 movie slide (source: Pixabay)

For more than 15 years, my colleagues and I at the Software Improvement Group (SIG) have been in the business of evaluating the quality of code, architecture, and development practices for our customers.

Recently, we dove into our assessment data to discover—and quantify—trends in software development, each time comparing 2016 to 2017. Below, we summarize our findings into five quantified trends, then discuss the developments that drive them. Spoiler alert: they are all about automation.

Learn faster. Dig deeper. See farther.

Join the O'Reilly online learning platform. Get a free trial today and find answers on the fly, or master something new and useful.

Learn more

Trend 1: The percentage of teams that do not automate deployment shrank from 26% in 2016 to 11% in 2017

We measure the practice of deployment automation by looking at whether teams have a deployment process in place that is quick, repeatable, and (preferably) fully automated.

For example, a team that deploys each new release with a single push of a button would receive a perfect score on this practice, codified as fully applied. But a team that needs to go through a limited number of well-documented manual steps would be scored as partially applied.

deployment automation
Figure 1. More teams fully apply deployment automation (43%, up from 30%) and fewer teams do not apply deployment automation at all (11%, down from 26%). Image by Joost Visser.

Trend 2: Teams that fully apply continuous integration (CI) now outnumber those that don’t (41% versus 32% in 2017; was 33% versus 39% in 2016)

The trend for continuous integration (automatic compilation and testing after each change) lags behind the trend for deployment automation, but overall, it shows a similar improvement.

deployment automation
Figure 2. Full or partial adoption of continuous integration (68% in 2017) has improved significantly but still lags compared to deployment automation (89% in 2017). Image by Joost Visser.

Trend 3: The number of teams that apply continuous delivery (CD) is a small but growing minority (16% in 2017 versus 11% in 2016)

Especially for continuous delivery (automatic deployment after each change), the great majority of teams (and the organizations of which they are part) still have a long way to go. But their numbers are growing.

deployment automation
Figure 3. Most teams still do not apply continuous delivery, either fully (16%; was 11%) or partially (12%; was 6%). Image by Joost Visser.

Trend 4: More teams are enforcing full quality control (31% in 2017, up from 23% in 2016)

To assess code quality control, we observe whether a team works with clear coding standards, systematically reviews code (against these standards and against principles of good design), and whether they perform automated code quality checks. Full adoption of code quality control has increased somewhat (31%, up from 23%), but 20% of teams are still producing code without adequate quality control in place.

deployment automation
Figure 4. Fewer teams are failing to enforce consistent code quality control (20%, down from 25%). Image by Joost Visser.

Trend 5: The number of teams that change their code without proper regression testing is declining but was still a staggering 41% in 2017 (down from 48% in 2016)

To assess testing practices, we observe whether teams have an automated regression test suite that is being executed consistently after each change. Full adoption of this practice is increasing (33%, up from 29%). But changing code without proper regression testing is still uncannily common (41%; was 48%).

deployment automation
Figure 5. Fewer teams fail to run automated test at each commit (41%, down from 48%). Image by Joost Visser.

So what do all of these numbers mean?

End-to-end automation of the software development process has gone mainstream. While automation of individual programming tasks (compilation, testing, documentation, etc.) has been part and parcel of our discipline for many years, we now see that all modern software development teams are striving to automate as much of the software development process as possible.

Automation helps to increase development speed, limit knowledge dissipation, and build quality into every step. If your team isn’t automating, now is the time to start.

If you’d like to learn where your team stands in terms of the above practices and trends, you can do a quick self-assessment by taking our survey.

This post is a collaboration between O’Reilly and SIG. See our statement of editorial independence.

Post topics: Software Architecture