Applied Measurement in Education · 1996 · 103 citations · 16 references
EducationEducational AssessmentEducational MeasurementGeneralizability TheoryNorm-referenced AssessmentPsychometricsInternal ConsistencyMeasurement ErrorMeasurement PrecisionMeasurement ValidityReliabilityTest ValidityGeneralizability IndexNorm-referenced Test InterpretationPrecision AnalysisPrecision AssessmentPrecision EstimationStandard Error Of MeasurementStandard Errors Of MeasurementTest Score Precision
Precision is defined as the absence of random error, but standard errors depend on score scale, making evaluation difficult without a benchmark, and tolerance for error—based on intended use—provides a context‑specific benchmark linking precision to validity. The study proposes evaluating precision by comparing measurement errors to the tolerance for error in a specific context. Reliability coefficients compare the standard error to observed score variability to provide a scale‑independent benchmark. Reliability and generalizability coefficients are scale‑independent but tied to norm‑referenced interpretations of test scores.
Measurements are said to be precise to the extent that they are free of random error, or equivalently, to the extent that they are consistent across different observations for an object of measurement. Standard errors of measurement provide convenient indexes of consistency, but their magnitude depends on the score scale and, therefore, it is difficult to make evaluative judgments about standard errors without having some benchmark for comparison. Reliability coefficients address this need by comparing the standard error with the observed score variability. The ratios defining reliability coefficients and generalizability coefficients are independent of the scale but are strongly linked to a norm-referenced interpretation of test scores. A more fundamental way to evaluate precision is to compare errors of measurement with the tolerance for error in a particular context. The tolerance for error specifies how large errors can be before they interfere with the intended use of the measurement procedure and is based on an analysis of the requirements for precision in that context. Because the tolerance for error is a function of the intended interpretations and uses of the measurement procedure, precision is an integral part of validity.
16
Statistical Theories of Mental Test Scores
William W. Rozeboom, Frederic M. Lord, Melvin R. Novick et al. · American Educational Research Journal · 1969 · 8.1K citations
Statistical theories of mental test scores
1974 · 1.2K citations