Nature · 2023 · 843 citations · 43 references
ComputingComputer ScienceArtificial IntelligenceComputer VisionMachine LearningComputer-aided DiagnosisMedical Image AnalysisPattern RecognitionRepresentation LearningRetinal ImagingSelf-supervised LearningFoundation Model PretrainingGeneralizability ModelsLabelsMedical Image ClassificationModel AdaptationPredictive Model PerformanceRetinal Image ProcessingSelf-supervised Image Representations
AI can detect health conditions from retinal images, yet existing models demand extensive annotation and are often task‑specific, limiting their generalizability. The study introduces RETFound, a foundation model that learns generalizable representations from unlabelled retinal images to enable label‑efficient adaptation across multiple applications. RETFound is trained on 1.6 million unlabelled retinal images via self‑supervised learning and subsequently fine‑tuned for disease‑detection tasks using labeled data. Adapted RETFound consistently outperforms comparison models in diagnosing and prognosing sight‑threatening eye diseases and predicting systemic disorders such as heart failure and myocardial infarction, while requiring fewer labeled samples and reducing expert annotation effort.
Abstract Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders 1 . However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications 2 . Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.
43
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das et al. · 2017 · 21.6K citations
Dermatologist-level classification of skin cancer with deep neural networks
Andre Esteva, Brett Kuprel, Roberto A. Novoa et al. · Nature · 2017 · 13.8K citations · Full text
The UK Biobank resource with deep phenotyping and genomic data
Clare Bycroft, Colin Freeman, Desislava Petkova et al. · Nature · 2018 · 9.7K citations · Full text