Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophrenia

TLDR

Unpublished rating scales in clinical trials have been linked to an increased likelihood of significant findings. The study aimed to assess whether unpublished scales bias results in schizophrenia trials. The authors randomly selected 300 trials from the Cochrane Schizophrenia Register, identified all treatment–control comparisons using rating scales, determined each scale’s publication status, and recorded claims of significant treatment effects. Trials using unpublished scales were 37 % more likely to report superiority, a bias that rose to 94 % under a gold‑standard definition, and in non‑pharmacological studies one‑third of such claims would disappear with published scales, confirming that unpublished scales bias schizophrenia trials.

Abstract

Background A recent review suggested an association between using unpublished scales in clinical trials and finding significant results. Aims To determine whether such an association existed in schizophrenia trials. Method Three hundred trials were randomly selected from the Cochrane Schizophrenia Group's Register. All comparisons between treatment groups and control groups using rating scales were identified. The publication status of each scale was determined and claims of a significant treatment effect were recorded. Results Trials were more likely to report that a treatment was superior to control when an unpublished scale was used to make the comparison (relative risk 1.37 (95% C11.12–1.68)). This effect increased when a ‘gold-standard’ definition of treatment superiority was applied (RR 1.94 (95% C11.35–2.79)). In non-pharmacological trials, one-third of ‘gold-standard’ claims of treatment superiority would not have been made if published scales had been used. Conclusions Unpublished scales are a source of bias in schizophrenia trials.