Content Specificity of Expert Judgments in a Standard‐Setting Study

Abstract

This study investigated the comparability of Angoff‐based item ratings on a general education test battery made by judges from within‐content specialties and across content domains. Judges were from English, mathematics, science, and social studies specialties in teacher education programs in a midwestem state. Cutscores established from the judges’ratings of out‐of‐content items differed little from the cutscores set using the ratings made by the content specialists. Further, out‐of‐content ratings by judges were not more influenced by performance data than were the ratings provided by judges rating items within their content specialty. The degree to ‐which these results generalize to other content specialties needs to be investigated.