Efficient implementation of gaussian processes

Abstract

Neural networks and Bayesian inference provide a useful framework within which to solve regression problems. However their parameterization means that the Bayesian analysis of neural networks can be difficult. In this paper, we investigate a method for regression using Gaussian process priors which allows exact Bayesian analysis using matrix manipulations. We discuss the workings of the method in detail. We will also detail a range of mathematical and numerical techniques that are useful in applying Gaussian processes to general problems including efficient approximate matrix inversion methods developed by Skilling. 1 Introduction Neural networks and Bayesian inference have provided a useful framework within which to solve regression problems (MacKay 1992a) (MacKay 1992b). However due to the parameterization of a neural network, implementations of the Bayesian analysis of a neural network require either maximum aposteriori approximations (MacKay 1992b) or the evaluation of integrals u...