The Relative Appropriateness of Eight Measurement Models for Analyzing Scores from Tests Composed of Testlets

Abstract

It has been shown that fundamental assumptions associated with conventional one-factor measurement models are frequently violated in analyses of scores from a test composed of testlets. Eight different measurement models were conceptualized for this kind of situation, and the goodness of fit of each model was examined. Conventional essentially tauequivalent and congeneric models present worse model fit to data and overestimate the reliability when testlets are involved. The one-factor congeneric model with correlated error specifications seems to be the best measurement model for a test composed of testlets if dichotomously scored items are used as the unit of analysis. However, in estimating score reliability for tests composed of testlets, the one-factor essentially tauequivalent model with correlated error specifications also provides good estimates. Measurement models using passage (testlet) scores would be alternatives for analyzing scores from tests composed of testlets when passage (testlet) scores are used as the unit of analysis.