Comparison of a Randomized and Two Quasi-Experimental Designs in a Single Outcome Evaluation

Abstract

The authors assessed the impact of three designs (randomized experiment, nonequivalent control group design, regression discontinuity design) on estimates of effect size of a university-level freshman remedial writing program. Designs were implemented within the same context, same time frame, and with the same population. The 375 freshman participants were either randomly assigned or self-selected into specific evaluation groups, according to design protocols. The three designs led to highly similar effect size estimates of the impact of a semester of remedial writing on writing outcomes following standard freshman composition. Specific design features contributed to the convergence of effect size estimates across designs.