Developing Computer Resources to Automate Analysis of Students’ Explanations of London Dispersion Forces

Abstract

Computer-assisted analysis of students’ written responses to questions is becoming a possibility due to developments in technology. This could make such constructed response questions more feasible for use in large classrooms where multiple choice assessments are often considered a more practical option. In this study, we use a previously developed prompt and coding scheme to characterize students’ explanations of the origins of London dispersion forces in order to develop machine learning resources that can carry out such an analysis for large numbers of students. We found that by using large numbers of human coded student responses (N = 1,730) we could subsequently automatically characterize students’ responses at a high level of accuracy compared to human coders. Furthermore, these resources were developed using responses from several different groups of students across multiple institutions to ensure both that our resources can work well with students from different backgrounds and that these computer resources can detect the different ways in which students explain this phenomenon. Such resources may help instructors to administer more complex open-ended assessment tasks to larger numbers of students and analyze the responses capturing language corresponding to causal mechanistic reasoning. Instructors could then use this information to better support their students’ learning.