Examining the Dependability of Academic Achievement Measures for English Language Learners

Abstract

The dependability of academic achievement measures for English language learners (ELLs) is influenced by three facts: (a) Each ELL has unique strengths and weaknesses in each language mode (listening, speaking, reading, and writing) both in English and in his or her first language, (b) each test item poses a different set of linguistic demands that do not replicate across languages, and (c) raters may vary in their ability to properly interpret students' responses. Generalizability theory allows examination of the magnitude of score variation due to the language (or dialect) in which tests are administered. The interaction of student, item, and language (and dialect) is the most important source of score variation. The language in which students should be tested and the minimum number of items needed to produce dependable measures of academic achievement for ELLs may be different across communities within a broad group of native speakers of a language.