Does Generative Artificial Intelligence Improve the Academic Achievement of College Students? A Meta-Analysis

Abstract

The use of generative artificial intelligence (Gen-AI) to assist college students in their studies has become a trend. However, there is no academic consensus on whether Gen-AI can enhance the academic achievement of college students. Using a meta-analytic approach, this study aims to investigate the effectiveness of Gen-AI in improving the academic achievement of college students and to explore the effects of different moderating variables. A total of 28 articles (65 independent studies, 1909 participants) met the inclusion criteria for this study. The results showed that Gen-AI significantly improved college students’ academic achievement with a medium effect size (Hedges’s g = 0.533, 95% CI [0.408,0.659], p < .05). There were within-group differences in the three moderator variables, activity categories, sample size, and generated content, when the generated content was text ( g = 0.554, p < .05), and sample size of 21–40 ( g = 0.776, p < .05), the use of independent learning styles ( g = 0.600, p < .05) had the most significant improvement in college student’s academic achievement. The intervention duration, the discipline types, and the assessment tools also had a moderate positive impact on college students’ academic achievement, but there were no significant within-group differences in any of the moderating variables. This study provides a theoretical basis and empirical evidence for the scientific application of Gen-AI and the development of educational technology policy.