Advances in Data Analysis and Classification · 2024 · 71 citations · 31 references
ComputingDecision SciencesForecastingPredictive AnalyticsStatisticsForecast EvaluationForecasting MethodsPredictive ModelingProbabilistic ForecastingTime Series ForecastingTime Series PredictionAccuracyForecast Accuracy AnalysisForecast Performance EvaluationForecasting AccuracyForecasting Accuracy EvaluationMann - Whitney U TestPredictive Performance
Abstract A key point to assess statistical forecasts is the evaluation of their predictive accuracy. Recently, a new measure, called Rank Graduation Accuracy (RGA), based on the concordance between the ranks of the predicted values and the ranks of the actual values of a series of observations to be forecast, was proposed for the assessment of the quality of the predictions. In this paper, we demonstrate that, in a classification perspective, when the response to be predicted is binary, the RGA coincides both with the AUROC and the Wilcoxon-Mann–Whitney statistic, and can be employed to evaluate the accuracy of probability forecasts. When the response to be predicted is real valued, the RGA can still be applied, differently from the AUROC, and similarly to measures such as the RMSE. Differently from the RMSE, the RGA measure evaluates point predictions in terms of their ranks, rather than in terms of their values, improving robustness.
31
Statistical Methods in the Atmospheric Sciences
Philip H. Ramsey, Daniel S. Wilks · Technometrics · 1996 · 7.7K citations
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
Glenn W. Brier · Monthly Weather Review · 1950 · 5.3K citations
Business, Mathematics, Meteorology +10
Francis X. Diebold, Roberto S. Mariano · Journal of Business and Economic Statistics · 1995 · 4.8K citations