EE-Explorer: A Multimodal Artificial Intelligence System for Eye Emergency Triage and Primary Diagnosis

Abstract

PurposeTo develop a multimodal artificial intelligence (AI) system, EE-Explorer, to triage eye emergencies and assist in primary diagnosis using metadata and ocular images.DesignA diagnostic, cross-sectional, validity and reliability study.MethodsEE-Explorer consists of 2 models. The triage model was developed from metadata (events, symptoms, and medical history) and ocular surface images via smartphones from 2038 patients presenting to Zhongshan Ophthalmic Center (ZOC) to output 3 classifications: urgent, semiurgent, and nonurgent. The primary diagnostic model was developed from the paired metadata and slitlamp images of 2405 patients from ZOC. Both models were externally tested on 103 participants from 4 other hospitals. A pilot test was conducted in Guangzhou to evaluate the hierarchical referral service pattern assisted by EE-Explorer for unspecialized health care facilities.ResultsA high overall accuracy, as indicated by an area under the receiver operating characteristic curve (AUC) of 0.982 (95% CI, 0.966-0.998), was obtained using the triage model, which outperformed the triage nurses (P < .001). In the primary diagnostic model, the diagnostic classification accuracy (CA) and Hamming loss (HL) in the internal testing were 0.808 (95% CI 0.776-0.840) and 0.016 (95% CI 0.006-0.026), respectively. In the external testing, model performance was robust for both triage (average AUC, 0.988, 95% CI 0.967-1.000) and primary diagnosis (CA, 0.718, 95% CI 0.644-0.792; and HL, 0.023, 95% CI 0.000-0.048). In the pilot test in the hierarchical referral settings, EE-explorer demonstrated consistently robust performance and broad participant acceptance.ConclusionThe EE-Explorer system showed robust performance in both triage and primary diagnosis for ophthalmic emergency patients. EE-Explorer can provide patients with acute ophthalmic symptoms access to remote self-triage and assist in primary diagnosis in unspecialized health care facilities to achieve rapid and effective treatment strategies. To develop a multimodal artificial intelligence (AI) system, EE-Explorer, to triage eye emergencies and assist in primary diagnosis using metadata and ocular images. A diagnostic, cross-sectional, validity and reliability study. EE-Explorer consists of 2 models. The triage model was developed from metadata (events, symptoms, and medical history) and ocular surface images via smartphones from 2038 patients presenting to Zhongshan Ophthalmic Center (ZOC) to output 3 classifications: urgent, semiurgent, and nonurgent. The primary diagnostic model was developed from the paired metadata and slitlamp images of 2405 patients from ZOC. Both models were externally tested on 103 participants from 4 other hospitals. A pilot test was conducted in Guangzhou to evaluate the hierarchical referral service pattern assisted by EE-Explorer for unspecialized health care facilities. A high overall accuracy, as indicated by an area under the receiver operating characteristic curve (AUC) of 0.982 (95% CI, 0.966-0.998), was obtained using the triage model, which outperformed the triage nurses (P < .001). In the primary diagnostic model, the diagnostic classification accuracy (CA) and Hamming loss (HL) in the internal testing were 0.808 (95% CI 0.776-0.840) and 0.016 (95% CI 0.006-0.026), respectively. In the external testing, model performance was robust for both triage (average AUC, 0.988, 95% CI 0.967-1.000) and primary diagnosis (CA, 0.718, 95% CI 0.644-0.792; and HL, 0.023, 95% CI 0.000-0.048). In the pilot test in the hierarchical referral settings, EE-explorer demonstrated consistently robust performance and broad participant acceptance. The EE-Explorer system showed robust performance in both triage and primary diagnosis for ophthalmic emergency patients. EE-Explorer can provide patients with acute ophthalmic symptoms access to remote self-triage and assist in primary diagnosis in unspecialized health care facilities to achieve rapid and effective treatment strategies.