Journal of Quantitative Analysis in Sports · 2013 · 49 citations · 4 references
ComputingData ScienceForecastingPerformance MeasurementPredictive AnalyticsEnsemble MethodsRanking ProblemsAccuracyCross-validationEmpirical PredictionFriedman Rank TestNemenyi TestNull HypothesisPredictive PerformanceScore-based Ranking
In this paper, we empirically evaluate the predictive power of eight sports ranking methods. For each ranking method, we implement two versions, one using only win-loss data and one utilizing score-differential data. The methods are compared on 4 datasets: 32 National Basketball Association (NBA) seasons, 112 Major League Baseball (MLB) seasons, 22 NCAA Division 1-A Basketball (NCAAB) seasons, and 56 NCAA Division 1-A Football (NCAAF) seasons. For each season of each dataset, we apply 20-fold cross validation to determine the predictive accuracy of the ranking methods. The non-parametric Friedman hypothesis test is used to assess whether the predictive errors for the considered rankings over the seasons are statistically dissimilar. The post-hoc Nemenyi test is then employed to determine which ranking methods have significant differences in predictive power. For all datasets, the null hypothesis – that all ranking methods are equivalent – is rejected at the 99% confidence level. For NCAAF and NCAAB datasets, the Nemenyi test concludes that the implementations utilizing score-differential data are usually more predictive than those using only win-loss data. For the NCAAF dataset, the least squares and random walker methods have significantly better predictive accuracy at the 95% confidence level than the other methods considered.
4
Statistical Comparisons of Classifiers over Multiple Data Sets
Janez Demšar · 2006 · 11.2K citations
Joyce Shaffer · Annual Review of Psychology · 1995 · 1.9K citations
Sensitivity and Stability of Ranking Vectors
Timothy P. Chartier, Erich Kreutzer, Amy N. Langville et al. · SIAM Journal on Scientific Computing · 2011 · 57 citations