A multi-national study of reading and tracing skills in novice programmers

TLDR

Many students fail to program after introductory courses, possibly due to poor problem‑solving or a fragile grasp of basic principles and tracing. The study sought to test the hypothesis that weak tracing skills underlie programming failure by evaluating students from seven countries. Students were assessed on two tasks: predicting the outcome of short code snippets and selecting the correct completion for near‑complete code from multiple choices. Results showed widespread weakness, especially on the completion task, indicating that fragile tracing skills are a key prerequisite for problem‑solving.

Abstract

A study by a ITiCSE 2001 working group ("the McCracken Group") established that many students do not know how to program at the conclusion of their introductory courses. A popular explanation for this incapacity is that the students lack the ability to problem-solve. That is, they lack the ability to take a problem description, decompose it into sub-problems and implement them, then assemble the pieces into a complete solution. An alternative explanation is that many students have a fragile grasp of both basic programming principles and the ability to systematically carry out routine programming tasks, such as tracing (or "desk checking") through code. This ITiCSE 2004 working group studied the alternative explanation, by testing students from seven countries, in two ways. First, students were tested on their ability to predict the outcome of executing a short piece of code. Second, students were tested on their ability, when given the desired function of short piece of near-complete code, to select the correct completion of the code from a small set of possibilities. Many students were weak at these tasks, especially the latter task, suggesting that such students have a fragile grasp of skills that are a prerequisite for problem-solving.