Modeling Guessing Behavior: A Comparison of Two IRT Models

Abstract

This study compared the fit of the three-parameter model to that of the Ability Removing Random Guessing (ARRG) model (Waller, 1973) on data from a wide range of tests of cognitive ability in three repre sentative samples. Although both models were de signed to remove only the effects of random guessing, the results of this study indicated that the three-param eter model also makes an adjustment for partial- knowledge guessing. Fit of the three-parameter model with guessing parameters estimated at a constant value of 1 divided by the number of alternatives was com pared to fit with individually estimated guessing pa rameters. The latter were found to produce fit far su perior to those estimated at a constant value. A solution to the convergence problems often encoun tered with the three-parameter model is discussed.