Clinical calibration of DSM‐IV diagnoses in the World Mental Health (WMH) version of the World Health Organization (WHO) Composite International Diagnostic Interview (WMH‐CIDI)

TLDR

The study outlines the rationale, design, and analysis plan for clinical calibration of the WMH‑CIDI, aiming to assess consistency with SCID diagnoses rather than validity. Consistency is evaluated at aggregate and individual levels using sensitivity, specificity, Cohen’s κ, and AUC, with prediction equations derived from WMH‑CIDI symptom data to estimate SCID probabilities, followed by substantive analyses of prevalence and associations and multiple‑imputation adjustment of standard errors. A brief illustration demonstrates that predicted SCID estimates of prevalence and correlates can be as precise as direct SCID estimates across different sample designs. © 2004 Whurr Publishers Ltd.

Abstract

Abstract An overview is presented of the rationale, design, and analysis plan for the WMH‐CIDI clinical calibration studies. As no clinical gold standard assessment is available for the DSM‐IV disorders assessed in the WMH‐CIDI, we adopted the goal of calibration rather than validation; that is, we asked whether WMH‐CIDI diagnoses are ‘consistent’ with diagnoses based on a state‐of‐the‐art clinical research diagnostic interview (SCID; Structured Clinical Interview for DSM‐IV) rather than whether they are ‘correct’. Consistency is evaluated both at the aggregate level (consistency of WMH‐CIDI and SCID prevalence estimates) and at the individual level (consistency of WMH‐CIDI and SCID diagnostic classifications). Although conventional statistics (sensitivity, specificity, Cohen's κ) are used to describe diagnostic consistency, an argument is made for considering the area under the receiver operator curve (AUC) to be a more useful general‐purpose measure of consistency. In addition, more detailed analyses are used to evaluate consistency on a substantive level. These analyses begin by estimating prediction equations in a clinical calibration subsample, with WMH‐CIDI symptom‐level data used to predict SCID diagnoses, and using the coefficients from these equations to assign predicted probabilities of SCID diagnoses to each respondent in the remainder of the sample. Substantive analyses then investigate whether estimates of prevalence and associations when based on WMH‐CIDI diagnoses are consistent with those based on predicted SCID diagnoses. Multiple imputation is used to adjust estimated standard errors for the imprecision introduced by SCID diagnoses being imputed under a model rather than measured directly. A brief illustration of this approach is presented in comparing the precision of SCID and predicted SCID estimates of prevalence and correlates under varying sample designs. Copyright © 2004 Whurr Publishers Ltd.