APPLYING ITEM RESPONSE THEORY METHODS TO COMPLEX SURVEY DATA

Abstract

Item response theory (IRT) offers many advantages to researchers who need to quantify children's reading and writing abilities, and for this reason, IRT methods have been adopted in Statistics Canada's National Longitudinal Survey for Children and Youth. IRT methods have a long history in the field of psychometrics, and provide a model based method for characterizing both test items and subject abilities, and for generating predictions of individual abilities. For the most part, IRT methods implicitly assume independent and identically distributed (i.i.d.) observations, so that the application of these methods to complex surveys raises a number of issues. The paper will review basic IRT theory and provide a rationale for its use in complex surveys. NLSCY data will be used to illustrate various IRT issues, including point and variance estimates of item parameters, the potential for bias due to ignoring survey weights, biases in the distribution of ability predictors, and the dependence of this bias on test length.