Study results from journals with a higher impact factor are closer to “truth”: a meta-epidemiological study

Abstract

BACKGROUND: Scientists, physicians, and the general public legitimately expect scholarly publications to give true answers to study questions raised. We investigated whether findings from studies published in journals with higher Journal Impact Factors (JIFs) are closer to truth than findings from studies in less-cited journals via a meta-epidemiological approach. METHODS: We screened intervention reviews from the Cochrane Database of Systematic Reviews (CDSR) and sought well-appraised meta-analyses. We used the individual RCT study estimates' relative deviation from the pooled effect estimate as a proxy for the deviation of the study results from the truth. The effect of the JIF on the relative deviation was estimated with linear regression and with local polynomial regression, both with adjustment for the relative size of studies. Several sensitivity analyses for various sub-group analyses and for alternative impact metrics were conducted. RESULTS: In 2459 results from 446 meta-analyses, results with a higher JIF were on average closer to "truth" than the results with a lower JIF. The relative deviation decreased on average by -0.023 per JIF (95% CI -0.32 to -0.21). A decrease was consistently found in all sensitivity analyses. CONCLUSIONS: Our results indicate that study results published in higher-impact journals are on average closer to truth. However, the JIF is only one weak and impractical indicator among many that determine a studies' accuracy.