An approach to evaluate adherence to the theme and the argumentative structure of essays

Abstract

This paper presents an approach based on machine learning for automatic grading of essay adherence to the theme and the argumentative structure of essays. The work was done according to the evaluation model adopted in Brazil to verify the mastery of skills and abilities of students who have completed high school. From specific adherence to the theme and the argumentative structure of essays features and others that are able to capture general aspects of the text, we trained and measured the efficiency of a classification and regression models based on support vector machines. The accuracy level found with this approach shows its effectiveness, pointing out the strategy as promising since other improvements can be achieved, such as improve the set of features with more effective essay-argumentative analysis techniques. Furthermore, we demonstrate how normalization and class balancing techniques are essential to improve our results using the small dataset available for this task. This work also introduces a set of textual cohesion features adapted to Portuguese that showed to be promising for evaluating adherence to the theme and the argumentative structure of essays.