An Overview of Automated Scoring of Essays.

TLDR

Automated Essay Scoring uses computer algorithms to evaluate written prose, addressing time, cost, reliability, and generalizability concerns in writing assessment and attracting widespread interest from schools, universities, and testing organizations. The article aims to provide an overview of current approaches to Automated Essay Scoring. It reviews major AES systems—PEG, IEA, E‑rater, Criterion, IntelliMetric, MY Access, and BETSY—examining their features and challenges in both low‑stakes classroom and high‑stakes standardized testing contexts.

Abstract

Automated Essay Scoring (AES) is defined as the computer technology that evaluates and scores the written prose (Shermis & Barrera, 2002; Shermis & Burstein, 2003; Shermis, Raymat, & Barrera, 2003). AES systems are mainly used to overcome time, cost, reliability, and generalizability issues in writing assessment (Bereiter, 2003; Burstein, 2003; Chung & O’Neil, 1997; Hamp-Lyons, 2001; Myers, 2003; Page, 2003; Rudner & Gagne, 2001; Rudner & Liang, 2002; Sireci & Rizavi, 1999; http://people.emich.edu). AES continue attracting the attention of public schools, universities, testing companies, researchers and educators (Burstein, Kukich, Wolff, Lu, & Chodorow, 1998; Shermis & Burstein, 2003; Sireci & Rizavi, 1999). The main purpose of this article is to provide an overview of current approaches to AES. After describing the most widely used AES systems (i.e., Project Essay Grader (PEG), Intelligent Essay Assessor (IEA), E-rater and Criterion, IntelliMetric and MY Access, and Bayesian Essay Test Scoring System (BETSY)), main characteristics of these systems will be discussed and current issues regarding the use of them both in low-stakes assessment (in classrooms) and high-stakes assessment (as standardized tests) will be discussed in this article.