Oxford Bulletin of Economics and Statistics · 2019 · 60 citations · 13 references
Social SciencesDevelopment StudiesEconomicsComparative Effectiveness ResearchEconometricsExperimental DesignHypothesis TestingLongitudinal ResearchQuasi-experimental EvaluationResearch Impact AssessmentSelection BiasStudy DesignImpact Evaluation FrameworksOutcome Impact EvaluationPublication Bias Over TimeTreatment Effects Estimation
Abstract This paper examines how significance inflation has varied across time, methods and disciplines. Leveraging a unique data set of impact evaluations on 20 kinds of development programmes, I find that results from randomized controlled trials exhibit less significance inflation than results from studies using other methods. Further, randomized controlled trials have exhibited less significance inflation over time, but quasi‐experimental studies have not. There is no robust difference between results from researchers affiliated with economics departments and those from researchers affiliated with other predominantly health‐related departments. Overall, the biases found appear much smaller than those previously observed in other social sciences.
13
Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn · Psychological Science · 2011 · 6.7K citations
John M. Hoenig, Dennis M. Heisey · The American Statistician · 2001 · 1.9K citations