Effect of Rasch Calibration on Ability and DIF Estimation in Computer‐Adaptive Tests

Abstract

In a previous simulation study of methods for assessing differential item functioning (DIF) in computer‐adaptive tests (Zwick, Thayer, & Wingersky, 1993, 1994), modified versions of the Mantel‐Haenszel and standardization methods were found to perform well. In that study, data were generated using the 3‐parameter logistic (3PL) model and this same model was assumed in obtaining item parameter estimates. In the current study, the 3PL data were used but the Rasch model was assumed in obtaining the item parameter estimates, which determined the information table used for item selection. Although the obtained DIF statistics were highly correlated with the generating DIF values, they tended to be smaller in magnitude than in the 3PL analysis, resulting in a lower probability of DIF detection. This reduced sensitivity appeared to be related to a degradation in the accuracy of matching. Expected true scores from the Rasch‐based computer‐adaptive test tended to be biased downward, particularly for lower‐ability examinees