Quality of implementation: developing measures crucial to understanding the diffusion of preventive interventions

TLDR

Preventive programs risk losing effectiveness when implementation quality is not maintained during dissemination. The study proposes a methodology for measuring implementation quality in schools and reports pilot data testing its components. The authors assessed adherence, process quality, adaptation valence, teacher attitudes, and content understanding through observation and interview of 11 Life Skills Training teachers. Observation and interview revealed that teachers varied in adherence and quality, all made adaptations, and that experienced teachers adhered better, delivered more interactively, communicated goals more clearly, and made more positive adaptations, providing a basis for improving implementation quality.

Abstract

As prevention programs become disseminated, the most serious threat to effectiveness is maintaining the quality of implementation intended by the developers. This paper proposes a methodology for measuring quality of implementation in school settings and presents data from a pilot study designed to test several of the proposed components. These methods included assessments of adherence, quality of process, the positive or negative valence of adaptations, teachers' attitudes and teachers' understanding of program content. This study was conducted with 11 teachers who had varying degrees of experience who taught Life Skills Training. Observation and interview data were collected during visits to schools. Results suggest that quality of implementation can be measured through observation and interview. Teachers varied in adherence and quality of program delivery. All teachers made adaptations to the program. Experienced teachers were more likely to adhere to the curriculum, deliver it in a way that was more interactive and engaging to students, communicate the goals and objectives better, and make positive adaptations. The field can use these findings as the basis for exploring strategies for measuring and improving quality of implementation.