Effects of the Rating Process on the Construct Validity of Assessment Center Dimension Evaluations

Abstract

Abstract Assessment center research routinely finds that correlations among dimensions within exercises are stronger than correlations within dimensions across exercises (i.e., the exercise effect). A study was designed to examine whether the commonly observed exercise effect associated with assessment center ratings could be explained in terms of the rating process. One hundred undergraduate students participated in a simulated assessment center that included 2 exercises and 4 dimensions. Fourteen trained undergraduates participated as assessors. Assessment center dimensions were rated using either a within-exercise rating process in which an assessor rated all dimensions within 1 exercise or a within-dimension rating process in which an assessor rated 1 dimension across all exercises. It was hypothesized that a within-exercise rating process would result in exercise factors and a within-dimension rating process would result in dimension factors. Confirmatory factor analyses of the 2 rating processes strongly supported the hypotheses. Implications for the design of assessment centers and future research are discussed.