Toward a Conceptual Framework for Mixed-Method Evaluation Designs

TLDR

Evaluators of educational and social programs have increasingly used mixed‑method designs, yet this practice requires a theory to guide design and implementation. This study developed a mixed‑method conceptual framework and proposed recommended designs for each of five purposes. The framework was derived from theoretical literature and refined through analysis of 57 empirical evaluations, and its design elements address methods, phenomena, paradigmatic framework, and implementation criteria. The framework identifies five purposes—triangulation, complementarity, development, initiation, and expansion—and highlights misuse of triangulation, low integration of methods at data analysis, and prioritizes strategies for integrated data analysis.

Abstract

In recent years evaluators of educational and social programs have expanded their methodological repertoire with designs that include the use of both qualitative and quantitative methods. Such practice, however, needs to be grounded in a theory that can meaningfully guide the design and implementation of mixed-method evaluations. In this study, a mixed-method conceptual framework was developed from the theoretical literature and then refined through an analysis of 57 empirical mixed-method evaluations. Five purposes for mixed-method evaluations are identified in this conceptual framework: triangulation, complementarity, development, initiation, and expansion. For each of the five purposes, a recommended design is also presented in terms of seven relevant design characteristics. These design elements encompass issues about methods, the phenomena under investigation, paradigmatic framework, and criteria for implementation. In the empirical review, common misuse of the term triangulation was apparent in evaluations that stated such a purpose but did not employ an appropriate design. In addition, relatively few evaluations in this review integrated the different method types at the level of data analysis. Strategies for integrated data analysis are among the issues identified as priorities for further mixed-method work.