Situational judgment tests: The influence and importance of applicant status and targeted constructs on estimates of <scp>B</scp> lack– <scp>W</scp> hite subgroup differences

Abstract

Situational judgment tests ( SJT s) have become popular in recent years. Yet, there is little research on how the constructs targeted within SJT s influence standardized ethnic group differences. Further, most research in this area is subject to differential range restriction concerns that hinder understanding of the role of constructs. We report on scale‐level data from four jobs in which SJT s were part of the first major hurdle of selection, thus providing an analysis of how constructs might relate to standardized group differences when range restriction concerns are minimized. Results indicate that cognitively saturated scales were associated with d values of 0.56 and 0.76, while interpersonal items were associated with d values of 0.07, 0.20, and 0.50. Based on the obtained d s, we simulated hiring and obtained adverse impact ratios to help interpret these values of d for decision‐makers. Overall, we demonstrate the importance of examining constructs and gathering data at the applicant level of analyses when attempting to understand SJT d values (and any associated adverse impact). Practitioner points The constructs targeted by SJT s may influence ethnic group differences, such that cognitively‐related items/scales may lead to larger group differences than scales targeting interpersonal skills. It can be useful to compare SJT d s based on both job applicant samples and targeted constructs during test development and during consideration of alternative predictors.