Perspectives on Evidence-Based Research in Education—What Works? Issues in Synthesizing Educational Program Evaluations

TLDR

Educational program syntheses are increasingly policy‑relevant and must minimize bias because individual evaluations are few. The article aims to outline key methodological requirements—design, sample size, pre‑test adjustments, duration, unbiased outcomes, and balancing design, effect size, and study count—to ensure syntheses yield reliable, unbiased evidence of program effectiveness. The authors review methodological criteria—including study design, sample size, pre‑test adjustments, duration, unbiased outcome measures, and the trade‑off among design, effect size, and study count—to guide program evaluation syntheses.

Abstract

Syntheses of research on educational programs have taken on increasing policy importance. Procedures for performing such syntheses must therefore produce reliable, unbiased, and meaningful information on the strength of evidence behind each program. Because evaluations of any given program are few in number, syntheses of program evaluations must focus on minimizing bias in reviews of each study. This article discusses key issues in the conduct of program evaluation syntheses: requirements for research design, sample size, adjustments for pretest differences, duration, and use of unbiased outcome measures. It also discusses the need to balance factors such as research designs, effect sizes, and numbers of studies in rating the overall strength of evidence supporting each program.