Validity and topic generality of a writing performance test

Abstract

The goal of the current study was to examine the validity and topic generality of a writing performance test designed to place international students into appropriate ESL courses at a large mid-western university. Because for each test administration the test randomly rotates three academic topics integrated with listening and reading sources, it is necessary to investigate the extent to which the three topics are compatible in terms of difficulty and generality across a diverse group of examinees. ESL Placement Test (EPT) scores from more than 1,000 examinees were modeled using multinomial logistic regression. Possible explanatory variables were identified as the assigned writing topic, students' majors, and their scores on the Test of English as a Foreign Language (TOEFL). Results indicate that after controlling for general English proficiency as measured by the TOEFL, students' majors were not related to their writing performance; however, the different topics did affect performance. In light of test validity, the demonstrated topic effect argues against the comparability of the three topics. Nevertheless, the absence of effect from the interaction of essay topic and writers' majors supports the generality of each topic for examinees from a wide range of disciplinary areas. Study limitations and future research suggestions are discussed.