Causal Discovery with Models: Behavior, Affect, and Learning in Cognitive Tutor Algebra

Abstract

Non-cognitive and behavioral phenomena, including gaming the system, off-task behavior, and affect, have proven to be important for understanding student learning outcomes. The nature of these phenomena requires investigations into their causal structure. For example, given that gaming the system has been associated with poorer learning outcomes, would reducing such behavior improve outcomes? Answering this question requires an understanding of whether gaming the system is a cause of poor outcomes, rather than, for example, only sharing a common cause with factors influencing learning. Because controlled experiments to settle such causal questions are often costly or impractical, we employ algorithmic search for the structure of graphical causal models from non-experimental data. Using sensor-free, data-driven detectors of behavior and affect, this work extends Baker and Yacef’s notion of “discovery with models ” to incorporate causal discovery and reasoning, resulting in an approach we call “causal discovery with models. ” We explore a case study of this approach using data from Carnegie Learning’s Cognitive Tutor for Algebra and raise questions for future research. Keywords Discovery with models, causal discovery, graphical causal models, probabilistic graphical models, gaming the system, affect, off-task behavior, sensor-free detectors, intelligent tutoring systems, Cognitive Tutor, measurement. 1.