Evaluating the predictive accuracy of volatility models

TLDR

Financial volatility forecasts are typically evaluated using standard statistical loss functions such as mean‑squared error. This paper proposes an alternative evaluation framework that uses probability scoring rules tailored to the forecast user's decision problem. The framework lets users define target economic events, select a scoring rule, and identify forecast subsets, then transforms volatility forecasts into probability forecasts of those events for evaluation with the chosen rule and calibration tests. An exchange‑rate example shows that the choice of loss function directly influences forecast evaluation outcomes. © 2001 John Wiley & Sons, Ltd.

Abstract

Standard statistical loss functions, such as mean-squared error, are commonly used for evaluating financial volatility forecasts. In this paper, an alternative evaluation framework, based on probability scoring rules that can be more closely tailored to a forecast user's decision problem, is proposed. According to the decision at hand, the user specifies the economic events to be forecast, the scoring rule with which to evaluate these probability forecasts, and the subsets of the forecasts of particular interest. The volatility forecasts from a model are then transformed into probability forecasts of the relevant events and evaluated using the selected scoring rule and calibration tests. An empirical example using exchange rate data illustrates the framework and confirms that the choice of loss function directly affects the forecast evaluation results. Copyright © 2001 John Wiley & Sons, Ltd.