Interobserver Variation in Interpreting Chest Radiographs for the Diagnosis of Acute Respiratory Distress Syndrome

TLDR

The study aimed to assess the reliability of chest radiograph interpretation for ARDS by having two intensivists and a radiologist independently review 778 images from 99 patients. After brief pilot training, raters scored each radiograph for diffuse bilateral infiltrates indicating ARDS and pairwise interobserver agreement was calculated. Moderate agreement (κ 0.38–0.55) was observed among raters without consensus training, whereas consensus‑trained raters achieved excellent agreement (κ 0.72–0.88), indicating that consensus training is essential for reliable ARDS diagnosis. Authors Meade MO, Cook RJ, Guyatt GH, Groll R, Kachura JR, Bedard M, Cook DJ, Slutsky AS, Stewart TE; title: Interobserver variation in interpreting chest radiographs for the diagnosis of acute respiratory distress syndrome.

Abstract

To measure the reliability of chest radiographic diagnosis of acute respiratory distress syndrome (ARDS) we conducted an observer agreement study in which two of eight intensivists and a radiologist, blinded to one another's interpretation, reviewed 778 radiographs from 99 critically ill patients. One intensivist and a radiologist participated in pilot training. Raters made a global rating of the presence of ARDS on the basis of diffuse bilateral infiltrates. We assessed interobserver agreement in a pairwise fashion. For rater pairings in which one rater had not participated in the consensus process we found moderate levels of raw (0.68 to 0.80), chance-corrected ( κ 0.38 to 0.55), and chance-independent ( Φ 0.53 to 0.75) agreement. The pair of raters who participated in consensus training achieved excellent to almost perfect raw (0.88 to 0.94), chance-corrected ( κ 0.72 to 0.88), and chance-independent ( Φ 0.74 to 0.89) agreement. We conclude that intensivists without formal consensus training can achieve moderate levels of agreement. Consensus training is necessary to achieve the substantial or almost perfect levels of agreement optimal for the conduct of clinical trials. Meade MO, Cook RJ, Guyatt GH, Groll R, Kachura JR, Bedard M, Cook DJ, Slutsky AS, Stewart TE. Interobserver variation in interpreting chest radiographs for the diagnosis of acute respiratory distress syndrome.