Supply Chain Risk Management

245

13

Risk Measurement

Imagine a measurement system that, when working eectively, oers the

opportunity to reduce supply chain risk. Next, imagine the possible out-

comes when such a system fails to work as intended. A number of years ago

a consumer products company with $100 million in annual sales developed a

scorecard system to measure supplier performance. Besides creating a sys-

tem that was not validated and was less than professional in appearance,

many larger suppliers challenged their scores, particularly when the scores

were lower than what they received from their more sophisticated custom-

ers. e measurement system was such a nonstarter that it deterred the

company from moving forward with its supplier measurement objectives.

It also aected, and not in a good way, the company’s relationships with its

suppliers. Not much in the way of risk reduction occurred here.

Welcome to the world of measurement, a topic that can enhance or

impede a company’s risk management eorts. is chapter examines risk

measurement from a variety of perspectives. We rst discuss measure-

ment validity and reliability, something that is critical as companies create

new ways to evaluate risk. is is followed by a presentation of best- in-

class supplier performance measurement systems, quantied risk indexes,

and a system for measuring risk at the country level. Next, we present the

increasingly talked about subject of total cost measurement. e chapter

concludes with a set of emerging risk metrics.

RISK MEASUREMENT VALIDITY AND RELIABILITY

As supply chain risk management (SCRM) evolves as a discipline, it

almost goes without saying that measurement will play an integral part.

246 • Supply Chain Risk Management: An Emerging Discipline

Recall that measurement is one of the key risk enablers we introduced in

Chapter3. As we work with companies, we are seeing all kinds of new

measures, measurement models, and risk indexes emerging that are part

of the risk management process. A basic concept whenever measurement

plays a central role is to ask a simple question: Is the measure or model

valid and reliable?

Valid means that an indicator or model measures what it is supposed to

measure. If we had to replace the word valid with another word, that word

would be accurate. If a social scientist develops a scale to measure indi-

vidual happiness, for example, does that scale actually measure happiness?

In the risk arena, if a measure is supposed to measure the probability of

a supplier failing nancially, does the measure actually measure nancial

distress? Or, an index might translate risk scores into a system that assigns

red, yellow, or green risk indicators. Is the cuto value dening red versus

yellow actually where the cuto should be? If a supplier measure indicates a

supplier is high risk, is it really a higher risk compared with other suppliers?

We do not want to give the impression that validating a measure or

model is easy to do. Our concern is that far too oen risk measures and

indicators are developed but not suciently tested, usually because valida-

tion can be a time- consuming process. In the social sciences, and many

observers consider business to be a social science, researchers have to

address many kinds of measurement validity or risk having their work

rejected by external reviewers. Dierent kinds of validity can include

construct, convergent, face, internal, predictive, statistical conclusion,

content, criterion, and concurrent validity. Validity has many dimensions,

enough to give a person a serious headache.

A second important dimension of a risk measure is reliability. Reliability

is the extent to which a measure provides results that are consistent from

use to use. A watch could measure time (it has validity), but it could be

inaccurate as its battery wears down. Or, the same piece of equipment used

to measure blood pressure is not reliable if it gives dierent readings when

no real change in a person’s blood pressure occurred. Something that is

reliable means that we have condence in its use time and time again.

Possible problems with risk measures and indexes are similar to Type I

and Type II measurement errors in quality management. A risk measure

or index may be so sensitive that it raises a red ag when no unusual prob-

lem or risk exists (i.e., a false positive, or Type I error). Aer receiving

enough false warnings, trust in the system erodes as users become desen-

sitized to what the measure conveys. Another possible outcome is similar