Software Metrics, 3rd Edition

475

Chapter 11

Software Reliability

Measurement and Prediction

*

A

  -  discussed in Chapter 10

identied soware reliability as a key high-level attribute. In

Chapter7, although we did not refer to reliability explicitly, the main

examples treated soware “quality” as a surrogate for soware reliability.

So, it is not surprising that soware reliability has been the most exten-

sively studied of all the quality attributes. Quantitative methods for its

assessment date back to the early 1970s, evolving from the theory of hard-

ware reliability. In this chapter, we describe these methods, highlight-

ing their limitations as well as their benets. Building on this work, we

describe an approach to soware reliability assessment that can provide

us with truly accurate predictive measures, providing we have been able to

collect data about past failures. e approach to measuring soware reli-

ability described in this chapter can also be incorporated with the causal

modeling approach described in Chapter 7 to achieve improved decision

making by soware quality and test managers, even when there is mini-

mal data available.

We begin by introducing the basics of reliability theory. en we address

what has become known as the soware reliability growth problem: esti-

mating and predicting the reliability of a program as faults are identied

and attempts are made to x them. In the approach described here, no

individual technique is singled out for unreserved recommendation from

*

Including contributions from prior editions by Bev Littlewood (City University, London).

476 ◾ Software Metrics

many that have been proposed over the years. We explain in some detail

that we reserve judgment, because empirical observation suggests that no

such technique has been able to consistently give accurate results over dif-

ferent data sources. Instead, we emphasize the need to examine the accu-

racy of the actual reliability measures, obtained from several techniques

in a particular case, with a view to selecting the one (if any) that yields

trustworthy results.

It is important to note that no current methods can feasibly assure

soware systems with ultra-high reliability requirements. However, the

techniques we describe apply to the vast majority of the systems we build:

those with relatively modest reliability requirements. Our suggested

approach is more computationally intensive than simply adopting a single

technique, and it involves some novel statistical techniques. However, we

explain the new methods intuitively, and you can apply the methods in

practice by using commercially available soware tools. e result is reli-

ability measures that are known to be trustworthy; so, this approach is

well worthwhile.

11.1 BASICS OF RELIABILITY THEORY

e theory of soware reliability has its roots in the more general

theory of systems and hardware reliability; the hardware approaches

are described in many textbooks (e.g., Rausand and Hoyland 2004;

Modarres etal. 2010; Birolini 2007). We apply many of the basic con-

cepts of this general theory to soware reliability problems in the dis-

cussion in this chapter.

e basic problem of reliability theory is to predict when a system will

eventually fail. In hardware reliability, we are normally concerned with

component failures due to physical wear; they can be caused, for exam-

ple, by corrosion, shock, or overheating. Such failures are probabilistic

in nature; that is, we usually do not know exactly when something will

fail, but we know that the product eventually will fail; so, we can assign

a probability that the product will fail at a particular point in time. For

example, suppose we know that a hose will eventually dry out, and that

the average usage time for the hose is 3 years. In other words, we can

expect that some time around the 3-year mark, the hose will begin to

leak. e probability of failure of the hose does not go from 0 on day 1094

(1 day short of 3 years) to 1 on day 1095; rather, the probability may start

at 0 on day 1 and then increase slowly as we approach the 3-year mark.

We can graph these daily probabilities over time, and the shape of the