A Research Synthesis of the Effectiveness, Replicability, and Generality of the VaNTH Challenge‐based Instructional Modules in Bioengineering

Abstract

B ackground Between 2000 and 2006 the Vanderbilt, Northwestern, Texas and Harvard/MIT Engineering Research Center (VaNTH/ERC) developed, tested, and implemented a set of educational innovations based largely on the ideas presented in the book How People Learn (HPL) and an instructional design known as the the STAR Legacy Cycle. The motivation for this study was to synthesize the results of this work. Published and unpublished experimental and quasi‐experimental assessments were included in this synthesis. P urpose (H ypothesis ) The fundamental hypotheses tested were whether a set of modules involving challenge‐based instruction and other course innovations, often involving advanced computer‐based technologies, improved student performance in a variety of educational settings and student populations, and whether improvements could be achieved by instructors other than the developers of the innovations. D esign /M ethod Meta‐analysis of effects from thirty‐three separate modules in five courses in bioengineering domains was undertaken, along with three case studies. R esults Results from the experimental (randomized) and stronger‐quasi experimental studies revealed a weighted effect size of 0.655 ( p < 0.001). Studies using randomized designs produced smaller effects, and studies using measures of transfer and adaptive expertise to index outcomes produced larger effects. Analyses also revealed that the results can be replicated by instructors other than the developers of the modules, in a variety of student populations and educational settings, and at other institutions. C onclusions Overall, the challenge‐based modules and other innovations have moderate overall effects on improved student performance. They can be implemented successively by other instructors in a variety of educational settings and student populations.