Journal of Educational and Behavioral Statistics · 2019 · 65 citations · 25 references
Public HealthBiostatisticsExperimental DesignExperimental PsychologyMeta-analysisHypothesis TestingPower AnalysisScientific EvidenceStudy DesignEvaluationExperimental ReplicabilityExperimental Results InterpretationReplicabilityReplication StudiesReplication Study Design
The problem of assessing whether experimental results can be replicated is becoming increasingly important in many areas of science. It is often assumed that assessing replication is straightforward: All one needs to do is repeat the study and see whether the results of the original and replication studies agree. This article shows that the statistical test for whether two studies obtain the same effect is smaller than the power of either study to detect an effect in the first place. Thus, unless the original study and the replication study have unusually high power (e.g., power of 98%), a single replication study will not have adequate sensitivity to provide an unambiguous evaluation of replication.
25
Statistical Power Analysis for the Behavioral Sciences
Keith E. Muller, Jacob Cohen · Technometrics · 1989 · 83.9K citations
Chinese Physics C · 2014 · 6.8K citations · Full text
Natural Sciences, Physics, Cosmology +16
1,500 scientists lift the lid on reproducibility
Monya Baker · Nature · 2016 · 4.3K citations
Fixed- and random-effects models in meta-analysis.
Larry V. Hedges, Jack L. Vevea · Psychological Methods · 1998 · 2.8K citations