Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction

TLDR

AI models can produce unreliable predictions when encountering data outside their training distribution. This study demonstrates that conformal prediction can identify unreliable prostate biopsy diagnoses and grades. The authors trained a model on 7,788 digitized prostate biopsies from 1,192 men and tested it on 3,059 biopsies from 676 men, applying conformal prediction to flag uncertain cases. Conformal prediction reduced cancer diagnosis errors from 2 % to 0.1 %, flagged 22 % of predictions as unreliable on in‑lab data, detected systematic performance drops in small external datasets, and lowered atypical‑tissue errors from 25 % to 2 % while flagging 80 % of such cases, thereby enhancing patient safety.

Abstract

Abstract Unreliable predictions can occur when an artificial intelligence (AI) system is presented with data it has not been exposed to during training. We demonstrate the use of conformal prediction to detect unreliable predictions, using histopathological diagnosis and grading of prostate biopsies as example. We digitized 7788 prostate biopsies from 1192 men in the STHLM3 diagnostic study, used for training, and 3059 biopsies from 676 men used for testing. With conformal prediction, 1 in 794 (0.1%) predictions is incorrect for cancer diagnosis (compared to 14 errors [2%] without conformal prediction) while 175 (22%) of the predictions are flagged as unreliable when the AI-system is presented with new data from the same lab and scanner that it was trained on. Conformal prediction could with small samples ( N = 49 for external scanner, N = 10 for external lab and scanner, and N = 12 for external lab, scanner and pathology assessment) detect systematic differences in external data leading to worse predictive performance. The AI-system with conformal prediction commits 3 (2%) errors for cancer detection in cases of atypical prostate tissue compared to 44 (25%) without conformal prediction, while the system flags 143 (80%) unreliable predictions. We conclude that conformal prediction can increase patient safety of AI-systems.