Behavioral and Brain Sciences · 1982 · 1.1K citations · 153 references
Public HealthBehavioral SciencesPsychologyAssessment FairnessClinical PsychologyManuscript StudiesPeer ReviewPublication BiasPublication EthicsResearch MisconductStandards Of ReviewJournal Peer ReviewPeer Review EvaluationPeer Review In JournalismPeer-to-peer ReviewResearch ArticlesResearch JournalsReviewer AccountabilityReviewer Reliability
Interest in the adequacy and fairness of peer‑review has risen, yet direct research on its reliability, accountability, bias, and competence remains scarce. This study directly examined the peer‑review process by resubmitting previously published manuscripts to the journals that originally reviewed them. Twelve articles from high‑rejection psychology journals were anonymized and resubmitted to the same editors 18–32 months after publication. Only 3 of 38 reviewers detected the resubmissions; nine of the 12 manuscripts proceeded to review, with eight rejected, and 16 of 18 reviewers advised against publication citing serious methodological flaws.
Abstract A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables. The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices. With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.
153
Social Theory and Social Structure
Eva J. Ross, Robert Κ. Merton · The American Catholic Sociological Review · 1958 · 10.4K citations
Toward an experimental ecology of human development.
Urie Bronfenbrenner · American Psychologist · 1977 · 10.2K citations
Social Theory and Social Structure.
Kenneth E. Bock, Robert Κ. Merton · American Sociological Review · 1969 · 6.7K citations