A Comparison of Alternative Models for Testlets

Abstract

When tests are made up of testlets, standard item response theory (IRT) models are often not appropriate due to the local dependence present among items within a common testlet. A testlet-based IRT model has recently been developed to model examinees' responses under such conditions (Bradlow, Wainer, & Wang, 1999). The Bradlow, Wainer, and Wang model introduces separate testlet factors to account for this dependence and applies a common item discrimination parameter to both the general ability and testlet factor. This study investigates several alternative ways of accounting for local dependence that make different assumptions regarding the influence of testlet factors on item performance. The authors implement several Bayesian model selection criteria to compare models using several real test data sets that have a testlet structure. Results suggest that an alternative model in which separate discrimination parameters are applied to the general ability and testlet factors provides a better fit to these data despite its greater complexity. Index terms: item response theory, Bayesian model comparison, Markov chain Monte Carlo, testlets