9.1 OVERVIEW
This chapter begins the discussion of a framework for evaluating
Open Systems Dependability (OSD), with the emphasis on the
evaluation of the whole lifecycle of a system. This is in addition to
the traditional one on the system itself using dependability metrics
(cf., ISO/IEC 9126-1:2001 and ISO/IEC 25000:2005). The evaluation
determines not only how much but also how well dependability is
achieved by examining the description of what the lifecycle of the
system accomplishes as well as how and why. The description to be
examined is an assurance case; in particular, it is a D-Case when the
lifecycle is the DEOS process.
The proposed evaluation framework divides its focus between
the substance of the lifecycle processes of a system and the quality
of what they produce; the latter includes not only the system but
also assurance cases at their various stages of existence. This split is
similar to that between ‘process arguments’ and ‘product arguments’
commonly found in assurance cases. There, a process argument may
show that, e.g., processes generate trustworthy evidence, while a
product argument may directly show from the evidence that, e.g.,
residual risks for the product are acceptably low. The two aspects
have further connections and importance in the evaluation of Open
Systems Dependability: development and operation processes are
a part of a ‘system (product)’ that is required to achieve OSD as a
whole; D-Cases (assurance cases) produced in the processes are
TOWARD AN OPEN SYSTEMS DEPENDABILITY
TOWARD AN OPEN SYSTEMS DEPENDABILITY
EVALUATION FRAMEWORK
EVALUATION FRAMEWORK
9
9
Toward an Open Systems Dependability Evaluation Framework
Toward an Open Systems Dependability Evaluation Framework 125
incorporated in the systems and controls parts of their operation
processes.
Evaluation of open aspects of a system should be an integral part
of that of all other aspects. The framework delineates certain focuses
of OSD evaluation so that it combines with other types of evaluation
without undue duplication of work. OSD evaluations may tell what
traditional benchmarking is justifi ably suffi cient; risk evaluations
at a higher level may tell what part of systems can be taken to be
closed. The framework is to be refi ned with such interfacing issues
with existing types of evaluations, but this chapter concentrate on
key aspects of OSD evaluation itself.
Section 9.2 organizes the evaluation of the lifecycle processes of
a system into that of four ‘process views’:
1. consensus building,
2. accountability achievement,
3. failure response,
4. change accommodation.
Figure 3-1 of the DEOS process depicts those four items variously
as key phases or cycles: it shows what the main concerns of activities
are when the activities are organized in the sequences and cycles.
However, those concerns cut across processes employed across the
lifecycle; i.e., they are addressed neither by a single process nor at
a single phase. Thus, a process view for each concern is required
to be defi ned, which gathers all relevant activities in the lifecycle
of a system, cf., (ISO/IEC 15288:2008, ISO/IEC 12207:2008). The
evaluation is on how well these process views achieve their desired
outcomes.
Section 9.3 specifi es four aspects of assurance case documents
as the focuses of evaluation:
1. internal consistency,
2. inter-system consistency,
3. soundness in the real world,
4. adequacy of degree confi dence.
Here, assurance cases are taken to contain all relevant descriptions
on artefacts produced throughout the lifecycle of a system, including
the ones on the lifecycle itself. The evaluation here determines the

Get Open Systems Dependability now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.