Frontiers in Education · 2023 · 35 citations · 64 references
EducationEducational AssessmentTeacher EducationFormative AssessmentNatural Language ProcessingScience Teacher EducationFormative Assessment In ScienceLearning Analytics For Formative AssessmentNatural Language Processing In EducationNatural Language UnderstandingReflective WritingWriting AssessmentAutomated Writing EvaluationFine-tuningFormative Assessment SoftwareFormative Language AssessmentNatural Language Processing EducationPre-service Science Teacher EducationSummative Assessment RubricsWriting-to-learn Assignments
Introduction Science educators use writing assignments to assess competencies and facilitate learning processes such as conceptual understanding or reflective thinking. Writing assignments are typically scored with holistic, summative coding rubrics. This, however, is not very responsive to the more fine-grained features of text composition and represented knowledge in texts, which might be more relevant for adaptive guidance and writing-to-learn interventions. In this study we examine potentials of machine learning (ML) in combination with natural language processing (NLP) to provide means for analytic, formative assessment of written reflections in science teacher education. Methods ML and NLP are used to filter higher-level reasoning sentences in physics and non-physics teachers’ written reflections on a standardized teaching vignette. We particularly probe to what extent a previously trained ML model can facilitate the filtering, and to what extent further fine-tuning of the previously trained ML model can enhance performance. The filtered sentences are then clustered with ML and NLP to identify themes and represented knowledge in the teachers’ written reflections. Results Results indicate that ML and NLP can be used to filter higher-level reasoning elements in physics and non-physics preservice teachers’ written reflections. Furthermore, the applied clustering approach yields specific topics in the written reflections that indicate quality differences in physics and non-physics preservice teachers’ texts. Discussion Overall, we argue that ML and NLP can enhance writing analytics in science education. For example, previously trained ML models can be utilized in further research to filter higher-level reasoning sentences, and thus provide science education researchers efficient mean to answer derived research questions.
64
The Measurement of Observer Agreement for Categorical Data
J. Richard Landis, Gary G. Koch · Biometrics · 1977 · 78.9K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
UMAP: Uniform Manifold Approximation and Projection
Leland McInnes, John Healy, Nathaniel Saul et al. · The Journal of Open Source Software · 2018 · 9.6K citations · Full text
Zellig S. Harris · WORD · 1954 · 4.7K citations