Rating Tasks Versus Dimensions in Assessment Centers: A Psychometric Comparison

Abstract

Assessment centers (ACs) have been widely criticized on the basis of measurement problems throughout the literature dating back to 1982. This study investigates whether an alternative to the prevailing trait paradigm would provide a more sensible treatment of AC ratings. All data were obtained in a real-world AC from the behavioral responses of 187 participants. Two paradigms of assessment were compared in a repeated measures design. The first model treated the AC data as though they comprised situationally specific behavioral samples. The second, more traditional model treated the data as though they were indicative of trait-based responses. Using generalizability theory, factor analysis, and confirmatory factor analysis, both models demonstrated similar psychometric characteristics, although only data treated under the situationally specific model held a conceptual justification in this study. These findings suggest that the situationally specific task-based model presents a more appropriate means by which to treat AC ratings in practice.