Comparison of the diagnostic utility of physician-diagnosed with algorithm-defined stroke-associated pneumonia

Abstract

OBJECTIVE: Diagnosing stroke-associated pneumonia (SAP) is challenging and may result in inappropriate antibiotic use or confound research outcomes. This study evaluates the diagnostic accuracy of algorithm-defined versus physician-diagnosed SAP in 1088 patients who had dysphagic acute stroke from 37 UK stroke units between 21 April 2008 and 17 May 2014. METHODS: SAP in the first 14 days was diagnosed by a criteria-based algorithm applied to blinded patient data and independently by treating physicians. Patients in whom diagnoses differed were reassigned following blinded adjudication of individual patient records. The sensitivity, specificity, positive predictive value (PPV) and diagnostic OR of algorithmic and physician diagnosis of SAP were assessed using adjudicated SAP as the reference standard. Agreement was assessed using the κ statistic. RESULTS: Physicians diagnosed SAP in 176/1088 (16%) and the algorithm in 123/1088 (11.3%) patients. Diagnosis agreed in 885/1088 (81.3%) patients (κ 0.22 (95% CI 0.14 to 0.29)). On a blinded review, 129/1088 (11.8%) patients were adjudicated as patients with SAP. The algorithm and the physicians had high specificity (97% (95% CI 96% to 98%) and 90% (95% CI 88% to 92%), respectively) but only moderate sensitivity (72% (95% CI 64% to 80%) and 65% (95% CI 56% to 73%), respectively) in diagnosing SAP. The algorithm showed better PPV (76% (95% CI 67% to 83%) vs 48% (95% CI 40% to 55%)), diagnostic OR (80 (95% CI 42 to 136) vs 18 (95% CI 12 to 27)) and agreement (κ 0.70 (95% CI 0.63 to 0.78) vs 0.48 (95% CI 0.41 to 0.54)) than physician diagnosis with adjudicated SAP. CONCLUSIONS: Algorithm-based approaches can standardise SAP diagnosis for clinical practice and research. TRIAL REGISTRATION NUMBER: ISRCTN37118456; Post-results.