272 Chapter 9. Analysis and Evaluation of Modifiability: The SAAM Method
10.3.6 Overall evaluation
Similar to the example of the KWIC problem, we assigned weights to the scenarios
according to our expectation on how probable the scenario will happen. The metric
for the evaluation of a particular architecture on a given scenario is the number of
components needing modification over the total number of components. The result
of the evaluation is therefore an estimation of the expected cost of modifications on
the architecture.
Table 10.7 Evaluation of KFV architectures
Scenarios Architectures
No. Weight
data type
1 20 5/7 2/12 0 0
2 5 0 0 0 0
3 15 1/7 1/12 0 1/7
4 5 4/7 3/12 4/14 2/7
5 5 4/7 2/12 2/14 2/7
6 5 4/7 3/12 4/14 2/7
7 10 3/7 4/12 4/14 1/7
8 15 1/7 1/12 1/14 1/7
9 20 7/7 4/12 0 7/7
Overall 51.43 19.15 7.49 30.00
The result of comparison shows that shared data architecture is the most
expensive design to modify, while abstract data type and implicit invocation are the
best with respect to modifiability. Notice that, the overall result of comparison
heavily depends on two factors: the set of scenarios and the weights assigned to
each scenario.
Software Design Methodology 273
SAAM is a scenario-based software architectural analysis method suitable for
evaluation and analysis of modifiability. There are two types of input to the
analysis: the architectural designs and the quality requirements. The process of
SAAM analysis consists of the following activities:
(1) Development of scenarios. Scenarios describe the interactions between
stakeholders and the system. Quality requirements can be represented in a
more detailed context by a set of scenarios.
(2) Description of candidate architectures. Software architecture to be evaluated
and compared are represented in a form understood by all stakeholders
involved in the analysis.
(3) Classification of scenarios into two types: direct scenarios and indirect
scenarios. Direct scenarios are those that the architecture supports directly
without need of any modification to the system. Indirect scenarios are those
that the architecture does not support and requires modifications to support the
(4) Individual evaluation of indirect scenarios. Each indirect scenario is further
analysed to identify the required modifications of the components and the
costs of such modifications.
(5) Assessment of scenario interaction. A component may need modifications in a
number of scenarios. To support all these scenarios, the component needs to
be modified to satisfy all the requirements imposed by these scenarios. Such
situations are complicated and difficult to find a solution. The assessment of
scenario interaction identifies such scenario interactions.
(6) Overall evaluation. Weights are assigned to scenarios according to their
importance and costs of the modifications on the components are estimated so
that numerical comparison between architectural designs can be made with
minimum subjectivity.
In this chapter, we demonstrated how to analyse each individual architectural
design as well as the comparison of a collection of designs by using SAAM. Such
analysis and comparison does not simply tell us whether an architectural design is
good or bad, but also tells us where the problems are and in what scenarios the
problem may occur. This provides detailed information for software designers to
improve their designs so that the problems are solved, or to take other technical
and/or management actions to prevent the problem occurring or to reduce the
consequences when the problem does occur.

Get Software Design Methodology now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.