Discriminative Learning of Sum-Product Networks

Abstract

Discriminatively-trained probabilistic models often outperform their generative counterparts on challenging tasks in computer vision and NLP. Conditional random elds have been a predomi-nant approach, but adapting them to richer structures further complicates learning with approximate inference [4]. Deep Networks have typically required both a generative and discriminative objec-tive involving approximate inference. The discriminative objective is usually reected in the use of backpropagation through softmax layers or support vector machines over network variables. The Sum-Product Network is a promising new deep architecture that can perform fast, exact inference on a representation that is broader than existing tractable graphical models [5]. For the rst time, we combine the advantages of SPNs with the advantages of discriminative learn-ing. We demonstrate the conditions under which an SPN represents the conditional partition func-tion and provide a discriminative training algorithm. In this setting, SPN nodes that exclusively cover variables conditioned upon need not be consistent or complete. This admits a larger class of discriminative SPNs with speedy inference. We dene an SPN that models the joint probability of three disjoint sets of hidden, query, and given variables: H, Y, and X respectively. The conditional probability of the query given the data is naturally P (Y jX) = P (Y;X) P (X) = P h