Standards for talking and thinking about validity.

Abstract

Standards for talking and thinking about validity have been promulgated in North America for decades. In 1954 two foundational standards were announced: (a) Thou shalt not refer to "the validity of the test" and (b) thou shalt use validity modifier labels, such as "content validity" or "predictive validity." Subsequently, in 1985, the latter became, thou shalt not use validity modifier labels. These standards for talking about validity have repeatedly been disregarded over the years. Possible reasons include intentional misuse, while upholding standards for thinking about validity; lack of awareness or misunderstanding of standards for thinking about validity; and genuine divergence from standards for thinking about validity. A historical analysis of disregard for these standards provides a basis for reappraising the concept of validity. We amassed a new body of evidence with which to challenge the frequently asserted claim that a general consensus exists over the meaning of validity. Indeed, the historical analysis provides reason to believe that prospects for achieving consensus over the meaning of validity are low. We recommend that the concept of validity be abandoned in favor of the more general, all-encompassing concept of quality, to be judged in relation to measurement aims, decision making aims, and broader policy aims, respectively.