Collabra Psychology · 2020 · 96 citations · 15 references
PharmacyBiostatisticsEconometricsOptimizationPredictive AnalyticsBias-variance TradeoffEstimation TheoryModel EvaluationModel FittingParameter EstimationVariance AnalysisAdjusted R-squaredBias-variance Tradeoff AnalysisEstimation Of Regression ModelsLinear RegressionMean Squared ErrorMean Squared Error EstimationModel Evaluation MetricsMultiple Regression
The amount of variance explained is widely reported for quantifying the model fit of a multiple linear regression model. The default adjusted R-squared estimator has the disadvantage of not being unbiased. The theoretically optimal Olkin-Pratt estimator is unbiased. Despite this, it is not being used due to being difficult to compute. In this paper, I present an algorithm for the exact and fast computation of the Olkin-Pratt estimator, which facilitates its use. I compare the Olkin-Pratt, the adjusted R-squared, and 18 alternative estimators using a simulation study. The metrics I use for comparison closely resemble established theoretical optimality properties. Importantly, the exact Olkin-Pratt estimator is shown to be optimal under the standard metric, which considers an estimator optimal if it has the least mean squared error among all unbiased estimators. Under the important alternative metric, which aims for the estimator with the lowest mean squared error, no optimal estimator could be identified. Based on these results, I provide careful recommendations on when to use which estimator, which first and foremost depends on the choice of which metric is deemed most appropriate. If such a choice is infeasible, I recommend using the exact Olkin-Pratt instead of the default adjusted R-squared estimator. To facilitate this, I provide the R package altR2, which implements the Olkin-Pratt estimator as well as all other estimators.
15
R: A Language and Environment for Statistical Computing
R Core Team · 2000 · 353K citations · Full text
Natural Sciences, Computer Science, Statistical Computing +10
Eric R. Ziegel, E. L. Lehmann, George Casella · Technometrics · 1999 · 4.3K citations
Redefine statistical significance
Daniel J. Benjamin, James O. Berger, Magnus Johannesson et al. · Nature Human Behaviour · 2017 · 2.8K citations · Full text
Statistical Science · 2010 · 2.4K citations · Full text