Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems

TLDR

The continuous ranked probability score (CRPS) was introduced as a verification tool for probabilistic forecast systems, encompassing the full range of a weather parameter, equivalent to an infinite‑class ranked probability score or the integral of the Brier score, and reducing to mean absolute error for deterministic forecasts. This study demonstrates that for ensemble prediction systems the CRPS can be decomposed into reliability and resolution/uncertainty components analogous to the Brier score decomposition. The proposed decomposition can be applied to continuous probability forecasts by interpreting them as the limit of ensemble forecasts with infinitely many members. The reliability component relates to the ensemble rank histogram, while the resolution/uncertainty component connects to average ensemble spread and outlier behavior, as illustrated with the European Centre for Medium‑Range Weather Forecasts system.

Abstract

Some time ago, the continuous ranked probability score (CRPS) was proposed as a new verification tool for (probabilistic) forecast systems. Its focus is on the entire permissible range of a certain (weather) parameter. The CRPS can be seen as a ranked probability score with an infinite number of classes, each of zero width. Alternatively, it can be interpreted as the integral of the Brier score over all possible threshold values for the parameter under consideration. For a deterministic forecast system the CRPS reduces to the mean absolute error. In this paper it is shown that for an ensemble prediction system the CRPS can be decomposed into a reliability part and a resolution/uncertainty part, in a way that is similar to the decomposition of the Brier score. The reliability part of the CRPS is closely connected to the rank histogram of the ensemble, while the resolution/uncertainty part can be related to the average spread within the ensemble and the behavior of its outliers. The usefulness of such a decomposition is illustrated for the ensemble prediction system running at the European Centre for Medium-Range Weather Forecasts. The evaluation of the CRPS and its decomposition proposed in this paper can be extended to systems issuing continuous probability forecasts, by realizing that these can be interpreted as the limit of ensemble forecasts with an infinite number of members.