Risk Management Technology in Financial Services

Elsevier UK Jobcode:RTF Chapter:CH11-H8304 22-3-2007 5:11p.m. Page:211 Trimsize:165×234MM

Fonts used:Ocean Sans & Sabon Margins:Top:42pt Gutter:54pt Font Size:10/12 Text Width:30p6 Depth:47 Lines

Testing, backtesting, post-mortems and experimental methodology 211

experimentally. In these and other firms, where internal prediction seems to have

given good results is in:



Capturing employee sentiment on project deadlines, and



Pinpointing problems in product quality more accurately than other measures

can do.

Associated with issues related to model review and analysis, which have been

brought to the reader’s attention in this section, should be tests designed for qualitative

factors which are not part of the algorithm, but can have significant impact on final

results. This enlarges the scope of stress testing a financial model, and brings into the

picture the issue of direct top management responsibility.

11.5 Post-mortems, experimentation and war games

Post-mortems have proved an invaluable tool in testing strategic decisions, from

investments to trades and the control of exposure. They provide significant informa-

tion on what went right and wrong including the original decision.

Johnson & Johnson has a policy, and a system, which sees to it that two years after

the company made a big acquisition senior management would do a thorough post-

mortem examination. This involves the top of the company’s hierarchy and takes the

form of an executive audit.



The post-mortem session starts with the original projections and original reasons

for the deal.



Participants identify the executives who made the arguments for acquisition and

compare these arguments with how the deal worked out.

It is very instructive to perform post-mortems on all types of major projects as

long as one does not get carried away with them. The goal is not witch-hunting, but

lessons to be learned and identification of errors not to be repeated. The ‘no blames’

policy is important because most managers don’t like doing post-mortems on their

key investment decisions because they are haunted by them.

There is always an excuse for something that went wrong, and many senior man-

agers spend a lot of time telling their board how wonderful their projects were, and

how justified their capital expenditures should have been. But they did not turn out

that way, and a post-mortem is the best way of taking the proverbial long, hard look

at what brought different results, after these results are known.

Chief executives renowned for sound governance behave differently, and for good

reason. They require that hours are spent examining stupid blunders, including the

reasons for underperformance and opportunity costs. They look where the institution

they are in charges, makes or loses too much money.



Which trading activity?



Which business unit?



Which elements turned the tables?



Was this adverse reaction foreseen?