Measuring Program Outcomes: Using Retrospective Pretest Methodology

TLDR

The study analyzed longitudinal data from 307 mothers in a home‑visitation child‑abuse prevention program. Retrospective pretest methodology yielded more accurate and legitimate program outcome assessments than traditional pretest‑posttest, avoiding underestimation due to response‑shift bias and offering a simple, convenient, expeditious evaluation approach. The study discusses limitations of retrospective pretests and methods to strengthen their use.

Abstract

This study used longitudinal data from 307 mothers with firstborn infants participating in a home-visitation, child-abuse prevention program. A self-report measure of specific constructs the program hoped to affect showed that the retrospective pretest methodology produced a more legitimate assessment of program outcomes than did the traditional pretest-posttest methodology. Results showed that when response shift bias was present, traditional pretest-posttest comparisons resulted in an underestimation of program effects that could easily be avoided by the retrospective pretest methodology. With demands for documenting program outcomes increasing, retrospective pretest designs are shown to be a simple, convenient, and expeditious method for assessing program effects in responsive interventions. The limits of retrospective pretests, and methods for strengthening their use, are discussed.