How Much Do the Effects of Education and Training Programs Vary Across Sites? Evidence From Past Multisite Randomized Trials

Abstract

Multisite trials, in which individuals are randomly assigned to alternative treatment arms within sites, offer an excellent opportunity to estimate the cross-site average effect of treatment assignment (intent to treat or ITT) and the amount by which this impact varies across sites. Although both of these statistics are substantively and methodologically important, only the first has been well studied. To help fill this information gap, we estimate the cross-site standard deviation of ITT effects for a broad range of education and workforce development interventions using data from 16 large multisite randomized controlled trials. We use these findings to explore hypotheses about factors that predict the magnitude of cross-site impact variation, and we consider the implications of this variation for the statistical precision of multisite trials.