Analysis of Student Behaviour in Habitable Worlds Using Continuous\n Representation Visualization

Abstract

We introduce a novel approach to visualizing temporal clickstream behaviour\nin the context of a degree-satisfying online course, Habitable Worlds, offered\nthrough Arizona State University. The current practice for visualizing\nbehaviour within a digital learning environment has been to generate plots\nbased on hand engineered or coded features using domain knowledge. While this\napproach has been effective in relating behavior to known phenomena, features\ncrafted from domain knowledge are not likely well suited to make unfamiliar\nphenomena salient and thus can preclude discovery. We introduce a methodology\nfor organically surfacing behavioral regularities from clickstream data,\nconducting an expert in-the-loop hyperparameter search, and identifying\nanticipated as well as newly discovered patterns of behavior. While these\nvisualization techniques have been used before in the broader machine learning\ncommunity to better understand neural networks and relationships between word\nvectors, we apply them to online behavioural learner data and go a step\nfurther; exploring the impact of the parameters of the model on producing\ntangible, non-trivial observations of behaviour that are suggestive of\npedagogical improvement to the course designers and instructors. The\nmethodology introduced in this paper led to an improved understanding of\npassing and non-passing student behaviour in the course and is widely\napplicable to other datasets of clickstream activity where investigators and\nstakeholders wish to organically surface principal patterns of behavior.