Remote image based retinopathy of prematurity diagnosis: a receiver operating characteristic analysis of accuracy

Abstract

BACKGROUND/AIMS: Telemedicine offers potential to improve the accessibility and quality of diagnosis of retinopathy of prematurity (ROP). The aim of this study was to measure accuracy of remote image based ROP diagnosis by three readers using receiver operating characteristic (ROC) analysis. METHODS: 64 hospitalised infants who met ROP examination criteria underwent two consecutive bedside procedures: dilated examination by an experienced paediatric ophthalmologist and digital retinal imaging with a commercially available wide angle camera. 410 images from 163 eyes were reviewed independently by three trained ophthalmologist readers, who classified each eye into one of four categories: no ROP, mild ROP, type 2 prethreshold ROP, or ROP requiring treatment. Sensitivity and specificity for detection of mild or worse ROP, type 2 prethreshold or worse ROP, and ROP requiring treatment were determined, compared to a reference standard of dilated ophthalmoscopy. ROC curves were generated by calculating values for each reader at three diagnostic cut-off levels: mild or worse ROP (that is, reader was asked whether image sets represented mild or worse ROP), type 2 prethreshold or worse ROP (that is, reader was asked whether image sets represented type 2 prethreshold or worse ROP), and ROP requiring treatment. RESULTS: Areas under ROC curves ranged from 0.747-0.896 for detection of mild or worse ROP, 0.905-0.946 for detection of type 2 prethreshold or worse ROP, and 0.941-0.968 for detection of ROP requiring treatment. CONCLUSIONS: Remote interpretation is highly accurate among multiple readers for the detection of ROP requiring treatment, but less so for detection of mild or worse ROP.