Making Software

Systematic Reviews in Software Engineering

In this section I present the results of some SRs that have challenged “common knowledge” and, in some cases, forced me to revise some of my ideas about software engineering.

Cost Estimation Studies

Cost estimation studies often report empirical studies of industry data sets, so research questions related to cost estimation would appear to be obvious candidates for SRs. Indeed, a review of SRs published between January 2004 and June 2007 found that cost estimation topics were the most common subject for SRs [Kitchenham et al. 2009b]. Two of these reviews are of particular interest because they overturn some of our preconceptions about software cost estimation.

The accuracy of cost estimation models

In two SRs, Magne Jørgensen addressed the issue of whether estimates from cost models (mathematical formulas usually generated from data collected on past projects) are more accurate than expert judgment estimates (estimates based on the subjective opinion of software developers or managers) [Jørgensen 2004], [Jørgensen 2007]. Since the publication of the books by Boehm [Boehm 1981] and DeMarco [DeMarco 1982] in the early 1980s, it has been an article of faith among cost estimation researchers that cost estimation models must be better than expert judgment, but Jørgensen’s reviews were the first attempt to determine whether this belief was supported by empirical evidence.

In [Jørgensen 2004], Jørgensen found 15 primary studies that compared expert ...