The sensitivity of belief networks to imprecise probabilities: an experimental investigation

TLDR

Bayesian belief networks are increasingly used for reasoning under uncertainty, yet obtaining precise numerical probabilities for large-scale applications is questioned. This study investigates how precise probabilities must be by measuring the impact of imprecision on diagnostic performance. Experiments on real-world medical diagnosis belief networks varied probability mappings, added random noise, simplified domain granularity, and tested cases with diseases outside the network. The results show that even highly imprecise input probabilities and binary representations only modestly reduce diagnostic accuracy, and that diseases outside the network slightly degrade performance, supporting belief networks as practical without requiring high precision.

Abstract

Bayesian belief networks are being increasingly used as a knowledge representation for reasoning under uncertainty. Some researchers have questioned the practicality of obtaining the numerical probabilities with sufficient precision to create belief networks for large-scale applications. In this work, we investigate how precise the probabilities need to be by measuring how imprecision in the probabilities affects diagnostic performance. We conducted a series of experiments on a set of real-world belief networks for medical diagnosis in liver and bile disease. We examined the effects on diagnostic performance of (1) varying the mappings from qualitative frequency weights into numerical probabilities, (2) adding random noise to the numerical probabilities, (3) simplifying from quaternary domains for diseases and findings—absent, mild, moderate, and severe—to binary domains—absent and present, and (4) using test cases that contain diseases outside the network. We found that even extreme differences in the probability mappings and large amounts of noise lead to only modest reductions in diagnostic performance. We found no significant effect of the simplification from quaternary to binary representation. We also found that outside diseases degraded performance modestly. Overall, these findings indicate that even highly imprecise input probabilities may not impair diagnostic performance significantly, and that simple binary representations may often be adequate. These findings of robustness suggest that belief networks are a practical representation without requiring undue precision.