Using Propensity Scoring to Estimate Program-Related Subgroup Impacts in Experimental Program Evaluations

Abstract

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions that program intake staff make about applicants' likely program assignments and experiences--that could improve the quality of matched comparison samples. They demonstrate the effectiveness of this approach in producing credible subgroup findings using data from a large-scale experimental evaluation of Job Corps, the nation's largest federal education and training program for disadvantaged youths.