The Role of Program Theory in Evaluation Research

Abstract

We examined the extent to which mathematics program evaluations that have been conducted according to methodologically rigorous standards have attended to the theories underlying the programs being evaluated. Our analysis focused on the 37 reports of K–12 mathematics program evaluations in the last two decades that have met standards for inclusion in What Works Clearinghouse syntheses. Each report was coded for the extent to which program theory was articulated and used in the evaluation, and for evaluator, study, and program characteristics. Our analyses revealed that very few evaluations attended to program theory, but that aspects of program theory use—as well as reported effect sizes—varied by study and evaluator characteristics.