Approximation Methods for Efficient Learning of Bayesian Networks

Abstract

Learning from data ranges between extracting essentials from the data, to the more fundamental and very challenging task of learning the underlying data generating process in terms of a probability distribution. In particular, in this thesis we assume that this distribution can be modelled as a Bayesian network. In terms of interpretability, the directed graphical structure (model) of a BN is attractive, because explicit insight is gained into relationships between variables. Most methods for learning require complete data in order to work or produce valid results. Unfortunately real-life databases are rarely complete. Learning from incomplete data is a non-trivial extension of existing methods developed for learning from complete data. In this thesis we develop and investigate efficient methods for learning Bayesian networks from both complete and incomplete data with emphasis on the latter. Several issues with regard to learning of Bayesian networks are discussed, including regularisation, and various scoring metrics are investigated and derived. Special attention is paid to a Bayesian statistical approach to learning. By way of Markov chain Monte Carlo (MCMC) simulation, Bayesian analysis has become a feasible alternative to the classical statistical approach. Different MCMC algorithms are presented for learning of Bayesian networks, both parameter and model, in a Bayesian statistical context. For large amounts of incomplete data where MCMC methods tend to be inefficient, approximations are implemented, such that learning remains feasible, albeit non-Bayesian. This leads to a very fast algorithm that outperforms existing approximate algorithms, and even competes with (structural) EM.