Market Risk Analysis Volume II: Practical Financial Econometrics

II.8

Forecasting and Model Evaluation

II.8.1 INTRODUCTION

Previous chapters in this volume have described econometric models for estimating and forecasting expected returns, volatility, correlation and multivariate distributions. This chapter describes how to select the best model when several models are available. We introduce the model specification criteria and model evaluation tests that are designed to help us choose between competing models, dividing them into two groups:

• Goodness-of-fit criteria and tests, which measure the success of a model to capture the empirical characteristics of the estimation sample. Goodness-of-fit tests are a form of in-sample specification tests.
• Post-sample prediction criteria and tests, which judge the ability of the model to provide accurate forecasts. Testing whether a forecast is accurate is a form of out-of-sample specification testing.

The second type of criterion or test is the most important. It is often possible to obtain an excellent fit within the estimation sample by adding more parameters to the model, but sometimes a tight fit on the estimation sample can lead to worse predictions than a loose fit. And it is the predictions of the model that really matter for market risk analysis. Analysing the risk and the performance in the past, when fitting a model to the estimation sample data, may provide some indication of the risk and the performance in the future. But we should never lose sight of the fact that portfolio risk is forward ...