Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology

TLDR

Replication studies that contradict prior findings can trigger self‑correction, yet the scientific community’s response to such results has not been systematically examined. This study investigated how citations to original psychology papers change after four high‑profile, multilaboratory replications that strongly contradicted and outweighed earlier findings. The authors assessed post‑replication citation patterns by tracking favorable and unfavorable citations to the original studies. The analysis revealed only a modest decline in favorable citations and a slight rise in unfavorable ones, with most citing articles ignoring the replications and about half of those that cited both the replication and the original offering explicit defenses, indicating limited corrective impact.

Abstract

Replication studies that contradict prior findings may facilitate scientific self-correction by triggering a reappraisal of the original studies; however, the research community’s response to replication results has not been studied systematically. One approach for gauging responses to replication results is to examine how they affect citations to original studies. In this study, we explored postreplication citation patterns in the context of four prominent multilaboratory replication attempts published in the field of psychology that strongly contradicted and outweighed prior findings. Generally, we observed a small postreplication decline in the number of favorable citations and a small increase in unfavorable citations. This indicates only modest corrective effects and implies considerable perpetuation of belief in the original findings. Replication results that strongly contradict an original finding do not necessarily nullify its credibility; however, one might at least expect the replication results to be acknowledged and explicitly debated in subsequent literature. By contrast, we found substantial citation bias: The majority of articles citing the original studies neglected to cite relevant replication results. Of those articles that did cite the replication but continued to cite the original study favorably, approximately half offered an explicit defense of the original study. Our findings suggest that even replication results that strongly contradict original findings do not necessarily prompt a corrective response from the research community.