License to evaluate

TLDR

Learning analytics bridges learning sciences and data analytics to exploit online learning data, with dashboards as common visual tools, yet prior work has not examined the theoretical foundations guiding their design and evaluation. This systematic review examines how learning‑science theories and models are incorporated into learner‑focused dashboard development. The authors systematically surveyed literature to assess the integration of learning‑science theories into dashboard design for learners. Few dashboard evaluations incorporate the underlying educational concepts, and peer‑comparison features are not universally well received, leading the authors to propose design and evaluation recommendations.

Abstract

Learning analytics can bridge the gap between learning sciences and data analytics, leveraging the expertise of both fields in exploring the vast amount of data generated in online learning environments. A typical learning analytics intervention is the learning dashboard, a visualisation tool built with the purpose of empowering teachers and learners to make informed decisions about the learning process. Related work has investigated learning dashboards, yet none have explored the theoretical foundation that should inform the design and evaluation of such interventions. In this systematic literature review, we analyse the extent to which theories and models from learning sciences have been integrated into the development of learning dashboards aimed at learners. Our analysis revealed that very few dashboard evaluations take into account the educational concepts that were used as a theoretical foundation for their design. Furthermore, we report findings suggesting that comparison with peers, a common reference frame for contextualising information on learning analytics dashboards, was not perceived positively by all learners. We summarise the insights gathered through our literature review in a set of recommendations for the design and evaluation of learning analytics dashboards for learners.