Organizational Research Methods · 2003 · 164 citations · 43 references
Public HealthBehavioral SciencesPsychologyExperimental DesignMeta-analysisPsychometricsModerator AnalysisPublication BiasSubgroup AnalysisSystematic ReviewMeta-analytic Bias AssessmentModeration AnalysisModerator BiasModerator ReportingResidual HeterogeneityStudy-level Moderators
This article reviews how moderators are detected and estimated in meta‑analysis and proposes recommendations based on that review and prior research. The authors examined all meta‑analyses published in the Journal of Applied Psychology between 1978 and 1997 to assess current moderator‑analysis practices. They found that moderator‑analysis practices vary widely, are often inconsistent with earlier guidance, and remain unchanged over time, prompting detailed recommendations for improvement.
The purpose of this article is to review current practices with respect to detection and estimation of moderators in meta-analysis and to develop recommendations that are driven by the results of this review and previous research. The first purpose was accomplished through a review of the meta-analyses published in Journal of Applied Psychology from 1978 to 1997. Results show, first, that practices with respect to both the execution of and the reporting of results from searches for moderators are highly variable and, second, that findings relevant for detection of moderators (e.g., percentage variance attributable to artifacts, SDρ, etc.) are often highly inconsistent with what has been suggested in the past. These practices held regardless of time of publication, specificity of the question addressed in the paper, and content area. Detailed suggestions for modifications of current practices are offered.
43