Using Latent Semantic Analysis to Evaluate the Contributions of Students in AutoTutor

Abstract

AutoTutor is a fully automated computer tutor that assists students in learning about hardware, operating systems, and the Internet in an introductory computer literacy course. AutoTutor presents questions and problems from a curriculum script, attempts to comprehend learner contributions that are entered by keyboard, formulates dialog moves that are sensitive to the learner's contributions (such as prompts, elaborations, corrections, and hints), and delivers the dialog moves with a talking head. Latent Semantic Analysis (LSA) is a major component of the mechanism that evaluates the quality of student contributions in the tutorial dialog. LSA's evaluations of college students' answers to deep reasoning questions are equivalent to the evaluations provided by intermediate experts of computer literacy, but not as high as more accomplished experts in computer science. LSA is capable of discriminating different classes of student ability (good, vague, erroneous or mute students) and in tracking the quality of contributions in tutorial dialog.