The Usefulness of the "Fail-Safe" Statistic in Meta-Analysis

Abstract

Rosenthal (1979, 1984) proposed that a fail-safe N statistic be calculated for all meta-analytic studies to aid researchers in detecting instability in meta-analytic results. However, few meta-analysts seem to have calculated fail-safe N's, despite the ease of doing so. As research exploring the practical utility of this statistic is nonexistent, this study evaluated the utility of fail-safe N's by computing them for studies in three research domains in which discrepant conclusions were reached by initial meta-analyses (with limited samples) and by subsequent follow-ups (with more inclusive samples). The current results suggest that calculation of a fail-safe N may have led to more cautious and circumspect interpretations of previous meta-analytic results. Implications of these findings for future meta-analytic research are discussed