From Clusters to Content

Abstract

Large undergraduate CS courses receive thousands of code submissions per term. To help make sense of the large quantities of submissions, projects have emerged to dynamically cluster student submissions by approach for writing scalable feedback, tailoring hints, and conducting research. However, relatively little attention has been paid to the value of these tools for informing revision to core course materials and delivery methods. In this work, we applied one such technology-Sense, the eponymous product of its company-to an online CS1 class delivered simultaneously for credit to on-campus students and for free to MOOC students. Using Sense, we clustered student submissions to around 70 problems used throughout the course. In this work, we discuss the value of such clustering, the surprising trends we discovered through this process, and the changes made or planned to the course based on the results. We also discuss broader ideas on injecting clustering results into course design.