Advanced Testing of Systems-of-Systems, Volume 2

6Metrics, KPI and Measurements

Reporting and monitoring progress means that the test manager must measure their project. This means simultaneously measuring the effectiveness and efficiency of testing activities, as well as the level of quality of the product, application, system or system-of-system being tested. So we will have two types of metrics:

– “project” metrics to measure the effectiveness and efficiency of activities against defined criteria;

– “product” metrics to determine the level of quality in relation to desired quality objectives, for example, defined on the basis of a risk analysis or contractual criteria.

Any evaluation must be objective; without objective measures, our management will be based on opinions or wishes, not on facts. By reusing metrics that we have previously defined, we are able to determine trends by comparing current measurements with the same measurements taken previously. The first measurement can constitute the baseline, that is, the reference measurement.

Metrics allow us to measure attributes and understand them. This will facilitate decision-making and will enable their relevance to be checked (by comparing objectives and reality). It is therefore necessary to select metrics and measures that allow stakeholders to make relevant decisions in a timely manner. Each project stakeholder will have different objectives from other stakeholders:

– customers are concerned by the availability of the application and its level of quality;

– hierarchy ...