“How We Know What We Know”: A Systematic Comparison of Research Methods Employed in Higher Education Journals, 1996−2000 v. 2006−2010

Abstract

This study replicates and extends a 2004 content analysis of three major higher education journals. The original study examined the methodological characteristics of all published research in these journals from 1996 to 2000, recommending that higher education programs adjust their graduate training to better match the heavily quantitative and statistically sophisticated journal content. We examine the same journals’ content from 2006 to 2010—one decade later—through the lens of knowledge production in higher education, and explore the ways that dominant modes of research may legitimize and/or delegitimize various forms of inquiry. Our findings reveal a field that continues to be dominated by quantitative methods and which is increasingly using more advanced statistical techniques. We discuss the tensions of a field more aligned with federal and state priorities and therefore better positioned to influence policy, but with a concomitantly contracted scope of and approach to inquiry. We also discuss implications for the training of graduate students, professionals, and policymakers as well as implications for publishing and researching other aspects of knowledge production in higher education.