Clinical Psychology Science and Practice · 1999 · 471 citations · 46 references
Public HealthBehavioral SciencesPsychologyComparative Effectiveness ResearchExperimental DesignPsychotherapyEfficacy EvaluationExperimental EvaluationModerator AnalysisStudy DesignSystematic ReviewTreatment EffectivenessComparative Efficacy EvidenceComparative PsychotherapyEfficacy ComparisonMeasurement Of Treatment EfficacyWild Cards
The study investigates how researchers’ treatment allegiances may bias comparative treatment outcomes. Researchers assessed allegiance using the traditional reprint method plus colleague and self‑ratings. Across multiple reviews, researchers’ allegiance consistently correlated with treatment outcomes, with combined measures explaining 69 % of variance, indicating a potential bias that should be addressed in comparative studies.
This report examines a possible distortion in the results of comparative treatment studies due to the association of the researcher's treatment allegiances with outcomes of those treatments. In eight past reviews a trend appeared for significant associations between the researcher's allegiance and outcomes of treatments compared. In a new review of 29 studies of treatment comparisons, a similar trend appeared. Allegiance ratings were based not only on the usual reprint method, but also on two new methods: ratings by colleagues who knew the researcher well, and self-ratings by the researchers themselves. The two new allegiance methods Interco related only moderately, but each allegiance measure correlated significantly with outcomes of the treatments compared, and when combined, the three measures explained 69% of the variance in outcomes Such an association can distort comparative treatment results. Our report concludes with how the researcher's allegiance may become associated with treatment outcomes and how studies should deal with these associations.
46
The file drawer problem and tolerance for null results.
Robert Rosenthal · Psychological Bulletin · 1979 · 8.6K citations
Comparative Studies of Psychotherapies
Lester Luborsky · Archives of General Psychiatry · 1975 · 1.5K citations