Item fit statistics for Rasch analysis: can we trust them?

Abstract

Abstract Aim To compare fit statistics for the Rasch model based on estimates of unconditional or conditional response probabilities. Background Using person estimates to calculate fit statistics can lead to problems because the person estimates are biased. Conditional response probabilities given the total person score could be used instead. Methods Data sets are simulated which fit the Rasch model. Type I error rates are calculated and the distributions of the fit statistics are compared with the assumed normal or chi-square distribution. Parametric bootstrap is used to further study the distributions of the fit statistics. Results Type I error rates for unconditional chi-square statistics are larger than expected even for moderate sample sizes. The conditional chi-square statistics maintain the significance level. Unconditional outfit and infit statistics have asymmetric distributions with means slighly below 1. Conditional outfit and infit statistics have reduced Type I error rates. Conclusions Conditional residuals should be used. If only unconditional residuals are available parametric bootstrapping is recommended to calculate valid p -values. Bootstrapping is also necessary for conditional outfit statistics. For conditional infit statistics the adjusted rule-of-thumb critical values look useful.