Criteria Teachers Use to Score Performance Items

Abstract

The purpose of this study was to validate rubric-related and investigate rubric-unrelated features used by teacher raters to assign scores to constructed-response items for mathematics and reading assessment. The 1,000 papers rated for the rubric-related and -unrelated features were selected randomly from a 10% sample of student responses in Kansas. The results validated the influence of correctness of answer in both the mathematics and reading holistic scores. In mathematics, the rubric-related reasoning feature was also supported, as reflected in the holistic scores. However, the other rubric-related feature in reading-irrelevant and inaccurate responses-was not supported. In addition, one possible threat to validity-length of response-was related to holistic scores even when rubric-related features were controlled. The role of response length may reflect differences between teachers and assessors in the perceived constructs measured by constructed-response items. Further research is needed not only to determine the contribution of length to holistic scores but also to examine the relations of teacher constructs concerning grades and holistic scores.