Programming trajectories analytics in block-based programming language learning

Abstract

Block-based programing languages (BBPL) provide effective scaffolding for K-12 students to learn computational thinking. However, the output-based assessment in BBPL learning is insufficient as we can not understand how students learn and what mistakes they have had. This study aims to propose a data-driven method that provides insight into students' problem-solving process in a game-based BBPL practice. Based on a large-scale programing dataset generated by 131,770 students in solving a classical maze game with BBPL in Hour of Code, we first conducted statistical analysis to extract the most common mistakes and correction trajectories students had. Furthermore, we proposed a novel program representation method based on tree edit distance of abstract syntax tree to represent students' programing trajectories, then applied a hierarchical agglomerative clustering algorithm to find the hidden patterns behind these trajectories. The experimental results revealed four qualitatively different clusters: quitters, approachers, solvers and knowers. The further statistical analysis indicated the significant difference on the overall performance among different clusters. This work provides not only a new method to represent students' programing trajectories but also an efficient approach to interpret students' final performance from the perspective of programing process.