The factor structure of test task characteristics and examinee performance

Abstract

The present study focuses on the task characteristics of reading passages and key sentences in a test of second language reading. Using a new methodological approach to describe variation in test task characteristics and explore how differences in these characteristics might relate to examinee performance, it posed the two following research questions: First, how do the characteristics of the texts used in a high-stakes test of English for Academic Purposes reading vary? Second, what relationships exist between its text characteristics and examinee performance? An expanded test task characteristics instrument was constructed, following Freedle and Kostin (1993) and Bachman et al. (1996), and adding a large number of syntactic features (Celce-Murcia and Larsen-Freeman, 1999). Ratings and numerical counts were compiled for three forms of the Test of English as a Foreign Language (TOEFL) Reading Comprehension Section. Taking items as the object of measurement, the results were then used in a series of exploratory and confirmatory factor analyses, along with IRT (item response theory) parameter estimates for the items in question.