Short, Simple, and Specific: The Influence of Item Design Characteristics in Multi‐Source Assessment Contexts

Abstract

The present study investigates the impact of item characteristics on multi‐source performance assessment. Three item characteristics (syntax, double‐barreledness, & behavioral specificity) were linked to the psychometric properties of items used by self, subordinates, peers, and supervisors as operationalized by the relationship between the item and the performance dimension it is intended to measure. Results show that syntax, a linguistic index that pertains to the length of items, is related to the psychometric properties of all rating sources except subordinates. The implications of this effect for the design of multi‐source assessment instrument are discussed.