Cross-Camera External Validation for Artificial Intelligence Software in Diagnosis of Diabetic Retinopathy

Abstract

Aims. To investigate the applicability of deep learning image assessment software VeriSee DR to different color fundus cameras for the screening of diabetic retinopathy (DR). Methods. Color fundus images of diabetes patients taken with three different nonmydriatic fundus cameras, including 477 Topcon TRC-NW400, 459 Topcon TRC-NW8 series, and 471 Kowa nonmyd 8 series that were judged as “gradable” by one ophthalmologist were enrolled for validation. VeriSee DR was then used for the diagnosis of referable DR according to the International Clinical Diabetic Retinopathy Disease Severity Scale. Gradability, sensitivity, and specificity were calculated for each camera model. Results. All images (100%) from the three camera models were gradable for VeriSee DR. The sensitivity for diagnosing referable DR in the TRC-NW400, TRC-NW8, and non-myd 8 series was 89.3%, 94.6%, and 95.7%, respectively, while the specificity was 94.2%, 90.4%, and 89.3%, respectively. Neither the sensitivity nor the specificity differed significantly between these camera models and the original camera model used for VeriSee DR development ( <a:math xmlns:a="http://www.w3.org/1998/Math/MathML" id="M1"> <a:mi>p</a:mi> <a:mo>=</a:mo> <a:mn>0.40</a:mn> </a:math> , <c:math xmlns:c="http://www.w3.org/1998/Math/MathML" id="M2"> <c:mi>p</c:mi> <c:mo>=</c:mo> <c:mn>0.065</c:mn> </c:math> , respectively). Conclusions. VeriSee DR was applicable to a variety of color fundus cameras with 100% agreement with ophthalmologists in terms of gradability and good sensitivity and specificity for the diagnosis of referable DR.