Automated writing evaluation: defining the classroom research agenda

TLDR

English is a global language, making strong writing skills essential worldwide, yet teaching writing is time‑intensive; automated writing evaluation (AWE) programs have been developed and psychometrically studied, but research on their classroom use remains scarce. The article aims to analyze recent AWE developments, explain their foundational principles, synthesize existing research, and propose a comprehensive research agenda for instructional use of AWE. The study reviews the scoring engines of online AWE programs, their psychometric analyses, and outlines potential methodologies and analytical tools for future research.

Abstract

With the advent of English as a global language, the ability to write well in English across diverse settings and for different audiences has become an imperative in second language education programmes throughout the world. Yet the teaching of second language writing is often hindered by the great amount of time and skill needed to evaluate repeated drafts of student writing. Online Automated Writing Evaluation programmes have been developed as a way to meet this challenge, and the scoring engines driving such programmes have been analysed in a considerable array of psychometric studies. However, relatively little research has been conducted on how AWE is used in the classroom and the results achieved with such use. In this article, we analyse recent developments in automated writing evaluation, explain the bases on which AWE systems operate, synthesize research with these systems, and propose a multifaceted process/product research programme on the instructional use of AWE. We explore this emerging area of inquiry by proposing a range of potential questions, methodologies and analytical tools that can define such a research agenda.