The existence and impact of floor effects for low-performing PISA participants

Abstract

Modern international studies of educational achievement have grown in terms of participating educational systems. Accompanying this development is an increase in heterogeneity, as more and different kinds of educational systems take part. This growth has been particularly pronounced among low-performing, less economically developed systems. Although studies such as PISA have made modifications to account for increased diversity, the degree to which international assessments serve educational systems at the lower ends of the achievement continuum is understudied. We used modified Wright maps and PISA’s definition of proficiency to evaluate the fitness of PISA, especially among low performers. Our findings suggest that there is mismatch between some populations and PISA. Results from a simulation show that such disparities produced biased achievement estimates and correlations with policy relevant variables. Projected PISA growth and new instantiations of PISA, particularly geared toward developing educational systems, make these findings timely and especially relevant.