Leveraging uncertainty information from deep neural networks for disease detection

TLDR

Deep learning has transformed computer vision and medical imaging, achieving expert‑level performance, yet disease‑detection models often lack uncertainty quantification, a capability clinicians rely on to decide when to seek additional expertise. The study evaluates dropout‑based Bayesian uncertainty estimates for diabetic retinopathy detection from fundus images and demonstrates that uncertainty‑guided referral improves diagnostic accuracy. The authors employ dropout‑based Bayesian inference on convolutional neural networks to estimate uncertainty, compare it to simpler methods, and analyze uncertainty sources via 2D visualizations mapped to high‑dimensional image space. Across multiple networks and datasets, the approach generalizes robustly, achieving over 85 % sensitivity and 80 % specificity when referring the top 20 % most uncertain cases, and shows that uncertainty is clinically relevant and can be made more robust to unfamiliar samples.

Abstract

Deep learning (DL) has revolutionized the field of computer vision and image processing. In medical imaging, algorithmic solutions based on DL have been shown to achieve high performance on tasks that previously required medical experts. However, DL-based solutions for disease detection have been proposed without methods to quantify and control their uncertainty in a decision. In contrast, a physician knows whether she is uncertain about a case and will consult more experienced colleagues if needed. Here we evaluate drop-out based Bayesian uncertainty measures for DL in diagnosing diabetic retinopathy (DR) from fundus images and show that it captures uncertainty better than straightforward alternatives. Furthermore, we show that uncertainty informed decision referral can improve diagnostic performance. Experiments across different networks, tasks and datasets show robust generalization. Depending on network capacity and task/dataset difficulty, we surpass 85% sensitivity and 80% specificity as recommended by the NHS when referring 0-20% of the most uncertain decisions for further inspection. We analyse causes of uncertainty by relating intuitions from 2D visualizations to the high-dimensional image space. While uncertainty is sensitive to clinically relevant cases, sensitivity to unfamiliar data samples is task dependent, but can be rendered more robust.