A variance explanation paradox: When a little is a lot.

Abstract

Concerning a single major league at bat, the percentage of variance in batting performance attributable to skill differentials among major league baseball players can be calculated statistically. The statistically appropriate calculation is seriously discrepant with intuitions about the influence of skill in batting performance. This paradoxical discrepancy is discussed in terms of habits of thought about the concept of variance explanation. It is argued that percent variance explanation is a misleading index of the influence of systematic factors in cases where there are processes by which individually tiny influences cumulate to produce meaningful outcomes. It is generally accepted that percentage of variance explained is a good measure of the importance of potential explanatory factors. Correlation coefficients of .30 or less are often poor-mouthed as accounting for less than 10% of the variance, a rather feeble performance for the influence of a putatively systematic factor. In analysis of variance contexts, the percentage of variance explanation is embodied in the omega-squared ratio of the systematic variance component to the total of the systematic and chance variance components. It, too, is often small; when it is, this is a source of discouragement for the thoughtful investigator.