Repair Theory: A Generative Theory of Bugs in Procedural Skills

TLDR

The paper proposes a generative theory that explains all procedural skill bugs as outcomes of constrained problem solving on incomplete procedures, and argues it can generalize beyond subtraction to procedural learning. The theory models incomplete learning and forgetting via formal deletion operations, tests this on a large multidigit subtraction bug database, and predicts bug occurrence, frequency, stability, and processing latencies without fitting data. The constrained solver and deletion operator successfully generate observed bugs while preventing impossible “star‑bugs,” and the model accurately predicts bug frequency, stability, and latency patterns.

Abstract

This paper describes a generative theory of bugs. It claims that all bugs of a procedural skill can be derived by a highly constrained form of problem solving acting on incomplete procedures. These procedures are characterized by formal deletion operations that model incomplete learning and forgetting. The problem solver and the deletion operator have been constrained to make it impossible to derive “star‐bugs”—algorithms that are so absurd that expert diagnosticians agree that the alogorithm will never be observed as a bug. Hence, the theory not only generates the observed bugs, it fails to generate star‐bugs. The theory has been tested on an extensive data base of bugs for multidigit subtraction that was collected with the aid of the diagnostic systems buggy and debuggy. In addition to predicting bug occurrence, by adoption of additional hypotheses, the theory also makes predictions about the frequency and stability of bugs, as well as the occurrence of certain latencies in processing time during testing. Arguments are given that the theory can be applied to domains other than subtraction and that it can be extended to provide a theory of procedural learning that accounts for bug acquisition. Lastly, particular care has been taken to make the theory principled so that it can not be tailored to fit any possible data.