Organization Science · 2010 · 134 citations · 81 references
Public HealthBehavioral SciencesPsychologyExperimental DesignMeta-analysisPsychometricsData AnalysisEffect Size EstimationHypothesis TestingStatistical AnalysisStatistical SignificanceTest ValidityAssessmentsFindings ValidationNull Hypothesis FormulationScientific HypothesesSignificance Criteria
Null-hypothesis significance tests (NHSTs) have received much criticism, especially during the last two decades. Yet many behavioral and social scientists are unaware that NHSTs have drawn increasing criticism, so this essay summarizes key criticisms. The essay also recommends alternative ways of assessing research findings. Although these recommendations are not complex, they do involve ways of thinking that many behavioral and social scientists find novel. Instead of making NHSTs, researchers should adapt their research assessments to specific contexts and specific research goals, and then explain their rationales for selecting assessment indicators. Researchers should show the substantive importance of findings by reporting effect sizes and should acknowledge uncertainty by stating confidence intervals. By comparing data with naïve hypotheses rather than with null hypotheses, researchers can challenge themselves to develop better theories. Parsimonious models are easier to understand, and they generalize more reliably. Robust statistical methods tolerate deviations from assumptions about samples.
81
Statistical Methods for Research Workers
American Journal of Clinical Pathology · 1945 · 11.1K citations
Why Most Published Research Findings Are False
John P. A. Ioannidis · PLoS Medicine · 2005 · 10.5K citations · Full text
Robust Regression and Outlier Detection
Gregory F. Piepel, Peter J. Rousseeuw, Annick M. Leroy · Technometrics · 1989 · 6.2K citations
The earth is round (p < .05).
Jacob Cohen · American Psychologist · 1994 · 3.9K citations