Educational Studies · 2025 · 35 citations · 19 references
EducationEducational AssessmentEducational TechnologyArtificial IntelligenceArtificial Intelligence In EducationAssessment DesignAutomated Essay ScoringCreativity AssessmentEssay WritingNatural Language Processing In EducationWriting AssessmentWriting SkillsAcademic Writing AssessmentAutomated Writing EvaluationComparative Data AnalysisCreative Writing AssessmentEducational Assessment ApplicationsEducational Creativity AssessmentMulti-tiered System FidelityRandomized Complete Block Design
This study examines the feasibility of AI in secondary school writing assessment by comparing GPT-4’s scoring performance with human raters. Using a randomized block design, 60 middle school students’ essays were evaluated by three experienced teachers and GPT-4. Results showed: (1) high overall agreement between AI and human raters (ICC = 0.84, 95% CI [0.79, 0.89]), with strongest correlation in high-level essays (r = 0.92, p < 0.001); (2) AI demonstrated superior efficiency, averaging 0.8 seconds per essay with stable performance (CV = 1.2%); (3) AI scored lower than humans in assessing creativity (M = 76.5 vs. 82.3, p < 0.05), revealing limitations in evaluating unconventional expressions. The study proposes a tiered system: AI for standardized scoring and humans for creative judgment. These findings support the cautious integration of AI in educational assessment..
19
Ismail Dergaa, Karim Chamari, Piotr Żmijewski et al. · Biology of Sport · 2023 · 741 citations · Full text