Making Software

The Effectiveness of TDD

We analyzed the TDD trials that reported quantitative results of the TDD pill’s effects on productivity, internal and external quality, and test quality. Direct comparison of the quantitative results across trials was impossible, since the trials measured TDD’s effectiveness in different ways. Instead, we assign each trial a summary value of “better,” “worse,” “mixed,” or “inconclusive/no-difference.” The summary value is determined by the quantitative results reported for the TDD pill compared with a control. The summary value also incorporates the report author’s interpretation of the trial results. In trials with a summary value of “better,” a majority of quantitative measures favor the TDD pill in comparison to the control treatment. In trials with a summary value of “worse,” a majority of measures favor the control treatment. Trials with a summary value of “inconclusive/no-difference” were inconclusive or report no observed differences. Finally, in trials with a summary value of “mixed,” some measures favor TDD while others don’t. In all cases, the summary assignation was guided by the report author’s interpretation of the study findings because, in many cases, the reports omitted details of the trial that would have enabled an objective external evaluation.

In the following sections we do our best to draw some conclusions about the value of TDD from the trials.

Internal Quality

Available evidence from the trials suggests that TDD does not have a consistent ...