Operational Risk: A Guide to Basel II Capital Requirements, Models, and Analysis

CHAPTER 10

Testing for the Goodness of Fit

The process of modeling operational losses is necessarily accompanied by model risk—the risk of selecting a wrong model. In operational risk modeling, correct model selection is critical because mistakes would have serious consequences on the amount of estimated VaR and the capital charge. An underestimated VaR would jeopardize the long-term ability of a bank to maintain a sufficient amount of capital in reserves to protect against catastrophic operational losses, while a severely overestimated VaR would limit the amount of funds available for investments. In both cases, a bank would be sending a bad signal to shareholders and other stakeholders.

Looking at the big picture, there are two types of procedures to test for the goodness of fit of the model:

1. In-sample goodness-of-fit tests.

2. Out-of-sample goodness-of-fit tests—that is, forecasting (backtesting).

Backtesting is an important procedure to test the validity of a chosen model and test its practical applicability.²²⁹We discuss backtesting of VaR models in Chapter 11. In practical applications, one cannot rely on one single test to determine which model is optimal, but instead should perform a variety of tests in model selection. In this chapter we focus on the in-sample goodness-of-fit tests. The mechanism of hypothesis testing is described in the appendix to this chapter.

VISUAL TESTS FOR THE GOODNESS OF FIT

Common visual tests for the goodness of fit include the quantile-quantile ...