LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE

TLDR

Complex, highly connected Bayesian belief networks can be conceptually and computationally intractable, so simpler models are preferable even if less accurate. The study presents a new approach for learning Bayesian belief networks from raw data. The method uses Rissanen's minimal description length principle, requires no prior distribution assumptions, can learn unrestricted multiply‑connected networks, and allows trading accuracy for complexity. The method generalizes prior Kullback–cross‑entropy approaches and experimental results demonstrate its feasibility.

Abstract

A new approach for learning Bayesian belief networks from raw data is presented. The approach is based on Rissanen's minimal description length (MDL) principle, which is particularly well suited for this task. Our approach does not require any prior assumptions about the distribution being learned. In particular, our method can learn unrestricted multiply‐connected belief networks. Furthermore, unlike other approaches our method allows us to trade off accuracy and complexity in the learned model. This is important since if the learned model is very complex (highly connected) it can be conceptually and computationally intractable. In such a case it would be preferable to use a simpler model even if it is less accurate. The MDL principle offers a reasoned method for making this trade‐off. We also show that our method generalizes previous approaches based on Kullback cross‐entropy. Experiments have been conducted to demonstrate the feasibility of the approach.