Computers and Education Artificial Intelligence · 2024 · 161 citations · 98 references
EducationComputer ScienceEducational TechnologyArtificial IntelligenceArtificial Intelligence In EducationHuman-centered DesignHuman-machine InteractionHuman-machine InterfacesLearning AnalyticsHuman-ai Collaboration In LearningHuman-ai InteractionHuman-automation InteractionHuman-centered AnalyticsHuman-centered Control SystemsHuman-centered Safety DesignHuman-computer Interaction In EducationSystematic Literature Review
Learning analytics and AI in education are rapidly expanding, yet concerns about data privacy, agency, and the lack of stakeholder involvement threaten trust and reveal gaps in understanding human control, safety, reliability, and trustworthiness. The authors conducted a systematic literature review to investigate these concerns and gaps. The review examined 108 papers to assess the state of human‑centred LA/AIED research, stakeholder participation, the balance between human control and automation, and the consideration of safety, reliability, and trustworthiness. The review found limited end‑user involvement in design despite some attention to human control, and recommends broader stakeholder participation, active student engagement, and prioritizing safety, reliability, and trustworthiness in future systems.
The rapid expansion of Learning Analytics (LA) and Artificial Intelligence in Education (AIED) offers new scalable, data-intensive systems but raises concerns about data privacy and agency. Excluding stakeholders—like students and teachers—from the design process can potentially lead to mistrust and inadequately aligned tools. Despite a shift towards human-centred design in recent LA and AIED research, there remain gaps in our understanding of the importance of human control, safety, reliability, and trustworthiness in the design and implementation of these systems. We conducted a systematic literature review to explore these concerns and gaps. We analysed 108 papers to provide insights about i) the current state of human-centred LA/AIED research; ii) the extent to which educational stakeholders have contributed to the design process of human-centred LA/AIED systems; iii) the current balance between human control and computer automation of such systems; and iv) the extent to which safety, reliability and trustworthiness have been considered in the literature. Results indicate some consideration of human control in LA/AIED system design, but limited end-user involvement in actual design. Based on these findings, we recommend: 1) carefully balancing stakeholders' involvement in designing and deploying LA/AIED systems throughout all design phases 2) actively involving target end-users, especially students, to delineate the balance between human control and automation, and 3) exploring safety, reliability, and trustworthiness as principles in future human-centred LA/AIED systems.
98
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Matthew J. Page, Joanne E. McKenzie, Patrick M. Bossuyt et al. · 2020 · 5.3K citations · Full text
The global landscape of AI ethics guidelines
Anna Jobin, Marcello Ienca, Effy Vayena · Nature Machine Intelligence · 2019 · 5K citations · Full text
ACM International Conference Proceeding Series
Gilberto Pedraza-García, Hernán Astudillo, Darío Correal · 2011 · 1.5K citations
The role of beliefs in teacher agency
Gert Biesta, Mark Priestley, Sarah Robinson · Teachers and Teaching · 2015 · 1.2K citations · Full text