Approximations for Binary Gaussian Process Classification

Abstract

We provide a comprehensive overview of many recent algorithms for approximate inference in\nGaussian process models for probabilistic binary classification. The relationships between several\napproaches are elucidated theoretically, and the properties of the different algorithms are\ncorroborated by experimental results. We examine both 1) the quality of the predictive distributions and\n2) the suitability of the different marginal likelihood approximations for model selection (selecting\nhyperparameters) and compare to a gold standard based on MCMC. Interestingly, some methods\nproduce good predictive distributions although their marginal likelihood approximations are poor.\nStrong conclusions are drawn about the methods: The Expectation Propagation algorithm is almost\nalways the method of choice unless the computational budget is very tight. We also extend\nexisting methods in various ways, and provide unifying code implementing all approaches.