Review of Research on Student-Facing Learning Analytics Dashboards and Educational Recommender Systems

TLDR

The review builds on four prior studies in similar domains. This article reviews student-facing learning analytics dashboards that report data directly to students. The authors screened 945 articles, included 93, and coded them across five categories—functionality, data sources, design analysis, student perceptions, and measured effects—while also examining design and development processes such as needs analyses, visual design, information selection, and student perception surveys. The review identifies gaps: research should target the design and development process of reporting systems, conduct experiments on effects on student behavior, achievement, and skills, incorporate usability tests and usage methodologies, and use observational methods like propensity score matching to rigorously measure effects.

Abstract

This article is a comprehensive literature review of student-facing learning analytics reporting systems that track learning analytics data and report it directly to students. This literature review builds on four previously conducted literature reviews in similar domains. Out of the 945 articles retrieved from databases and journals, 93 articles were included in the analysis. Articles were coded based on the following five categories: functionality, data sources, design analysis, student perceptions, and measured effects. Based on this review, we need research on learning analytics reporting systems that targets the design and development process of reporting systems, not only the final products. This design and development process includes needs analyses, visual design analyses, information selection justifications, and student perception surveys. In addition, experiments to determine the effect of these systems on student behavior, achievement, and skills are needed to add to the small existing body of evidence. Furthermore, experimental studies should include usability tests and methodologies to examine student use of these systems, as these factors may affect experimental findings. Finally, observational study methods, such as propensity score matching, should be used to increase student access to these systems but still rigorously measure experimental effects.