Making Software

Summary

In the preceding sections we have described various metrics that have been used to predict failures from the Windows experience. To summarize, Table 23-3 summarizes the precision and recall of using these sets of metrics to predict failures in Windows Vista.

Table 23-3. Overall model accuracy using different software measures

Model	Precision	Recall
Organizational structure	86.2%	84.0%
Code churn	78.6%	79.9%
Code complexity	79.3%	66.0%
Social network/combination measures	76.9%	70.5%
Dependencies	74.4%	69.9%
Code coverage	83.8%	54.4%

In addition, social network measures developed by integrating several of the measures also provides accurate predictions of failures. As stated earlier, as with all empirical studies, the evidence provided here is based only upon empirical results obtained at Microsoft, and more specifically in Windows. It is possible that these results might not hold for other software products or environments. We become more confident in results when similar studies are done in different software domains to show or disprove generality of the evidence provided for failure prediction. A step-by-step guide to building predictors is illustrated in the sidebar [Nagappan et al. 2006a]. The primary purpose and goal of this chapter is to share experiences of results learned at Microsoft using various metrics and to describe the metrics, to encourage repetition in different domains by researchers and software engineers and thereby determine the efficacy ...