Assessment of Differential Item Functioning for Performance Tasks

Abstract

Although the belief has been expressed that performance assessments are intrinsically more fair than multiple‐choice measures, some forms of performance assessment may in fact be more likely than conventional tests to tap construct‐irrelevant factors. The assessment of differential item functioning (DIF) can be helpful in investigating the effect on subpopulations of the introduction of performance tasks. In this study, two extensions of the Mantel‐Haenszel (MH; 1959) procedure that may be useful in assessing DIP in performance measures were explored. The test of conditional association proposed by Mantel (1963) seems promising as a test of DIF for pofytomous items when the primary interest is in the between‐group difference in item means, conditional on some measure of ability. The generalized statistic proposed by Mantel and Haenszel may be more useful than Mantel's test when the entire response distributions of the groups are of interest. Simulation results showed that, for both inferential procedures, the studied item should be included in the matching variable, as in the dichotomous case. Descriptive statistics that index the magnitude of DIP were also investigated.