“Let the community decide”? The vision and reality of soundness-only peer review in open-access mega-journals

Abstract

Purpose The purpose of this paper is to better understand the theory and practice of peer review in open-access mega-journals (OAMJs). OAMJs typically operate a “soundness-only” review policy aiming to evaluate only the rigour of an article, not the novelty or significance of the research or its relevance to a particular community, with these elements being left for “the community to decide” post-publication. Design/methodology/approach The paper reports the results of interviews with 31 senior publishers and editors representing 16 different organisations, including 10 that publish an OAMJ. Thematic analysis was carried out on the data and an analytical model developed to explicate their significance. Findings Findings suggest that in reality criteria beyond technical or scientific soundness can and do influence editorial decisions. Deviations from the original OAMJ model are both publisher supported (in the form of requirements for an article to be “worthy” of publication) and practice driven (in the form of some reviewers and editors applying traditional peer review criteria to OAMJ submissions). Also publishers believe post-publication evaluation of novelty, significance and relevance remains problematic. Originality/value The study is based on unprecedented access to senior publishers and editors, allowing insight into their strategic and operational priorities. The paper is the first to report in-depth qualitative data relating specifically to soundness-only peer review for OAMJs, shedding new light on the OAMJ phenomenon and helping inform discussion on its future role in scholarly communication. The paper proposes a new model for understanding the OAMJ approach to quality assurance, and how it is different from traditional peer review.