Learning Sum-Product Networks with Direct and Indirect Variable Interactions

Abstract

Sum-product networks (SPNs) are a deep prob-abilistic representation that allows for efficient, exact inference. SPNs generalize many other tractable models, including thin junction trees, latent tree models, and many types of mixtures. Previous work on learning SPN structure has mainly focused on using top-down or bottom-up clustering to find mixtures, which capture vari-able interactions indirectly through implicit la-tent variables. In contrast, most work on learning graphical models, thin junction trees, and arith-metic circuits has focused on finding direct in-teractions among variables. In this paper, we present ID-SPN, a new algorithm for learning SPN structure that unifies the two approaches. In experiments on 20 benchmark datasets, we find that the combination of direct and indirect interactions leads to significantly better accuracy than several state-of-the-art algorithms for learn-ing SPNs and other tractable models. 1.