Reliability Methods: A Note on the Frequency of Use of Various Types

TLDR

The study examined how often different reliability coefficients were reported for 696 tests in the APA Directory of Unpublished Experimental Mental Measures. The authors noted problems such as ambiguous coefficient labels, reliance on external studies, missing subscale data, and recording errors. Nearly all articles reported at least one reliability estimate, with coefficient alpha overwhelmingly preferred, while several textbook‑standard measures were rarely or never used.

Abstract

This study examined the frequency of use of various types of reliability coefficients for a systematically drawn sample of 696 tests appearing in the APA-published Directory of Unpublished Experimental Mental Measures. Almost all articles included some type of reliability report for at least one test administration. Coefficient alpha was the over-whelming favorite among types of coefficients. Several measures treated almost universally in psychological-testing textbooks were rarely or never used. Problems encountered in the study included ambiguous designations of types of coefficients, reporting reliability based on a study other than the one cited, inadequate information about subscales, and simply incorrect recording of the information given in an original source.