Knowledge and Information Systems · 2023 · 32 citations · 48 references
EducationComputer ScienceEducational AssessmentComputational LinguisticsIntelligent Tutoring SystemsNatural Language ProcessingFeedback In AssessmentIntelligent Tutoring System AnalyticsNatural Language Processing In EducationQuestion AnsweringStudent AssessmentTutoring System EvaluationAutomated Short-answer ScoringDataset CurationIntelligent Tutoring Systems EvaluationLanguage UnderstandingNatural Language InterfacesOpen-ended Response ScoringQuestion Answering SystemsSemantic Features
Abstract Automatic short answer grading (ASAG), a hot field of natural language understanding, is a research area within learning analytics. ASAG solutions are conceived to offload teachers and instructors, especially those in higher education, where classes with hundreds of students are the norm and the task of grading (short)answers to open-ended questionnaires becomes tougher. Their outcomes are precious both for the very grading and for providing students with “ ad hoc ” feedback. ASAG proposals have also enabled different intelligent tutoring systems. Over the years, a variety of ASAG solutions have been proposed, still there are a series of gaps in the literature that we fill in this paper. The present work proposes GradeAid , a framework for ASAG. It is based on the joint analysis of lexical and semantic features of the students’ answers through state-of-the-art regressors; differently from any other previous work, (i) it copes with non-English datasets, (ii) it has undergone a robust validation and benchmarking phase, and (iii) it has been tested on every dataset publicly available and on a new dataset (now available for researchers). GradeAid obtains performance comparable to the systems presented in the literature (root-mean-squared errors down to 0.25 based on the specific tuple $$\langle $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>⟨</mml:mo> </mml:math> dataset-question $$\rangle $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mo>⟩</mml:mo> </mml:math> ). We argue it represents a strong baseline for further developments in the field.
48
Interrater reliability: the kappa statistic
Marry L. McHugh · Biochemia Medica · 2012 · 18.6K citations · Full text
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Nils Reimers, Iryna Gurevych · 2019 · 10.9K citations · Full text
An Analysis of Variance Test for Normality (Complete Samples)
Samuel S. Shapiro, M. B. Wilk · Biometrika · 1965 · 5.9K citations
Hierarchical Attention Networks for Document Classification
Zichao Yang, Diyi Yang, Chris Dyer et al. · 2016 · 4.8K citations · Full text
Education and the COVID-19 pandemic
Sir John Daniel · Prospects · 2020 · 2.4K citations · Full text