Does a Standard Reflect Minimal Competency of Examinees or Judge Competency?

Abstract

Abstract This study examines the influence of judges' item-related knowledge on standard setting for competency tests. Seventeen judges took a 122-item high-school teacher certification test in economics while setting competency standards for the test using the Angoff procedure. Judges tended to set higher standards for items they answered correctly and lower standards for items they answered incorrectly. Standards were also more consistent for items judges answered correctly than for items judges answered incorrectly. Implications for standard-setting practice regarding the heterogeneity of judges' test-related knowledge are discussed.