Bayesian Interpolation

TLDR

Bayesian analysis has long been used, but systematic Bayesian model‑comparison has only recently been developed in depth, notably by Gull and Skilling. The paper demonstrates a Bayesian approach to regularization and model‑comparison by applying it to the inference problem of interpolating noisy data, and notes that the concepts are general for other data‑modeling problems. Regularizing constants are set by examining their posterior probability distribution, alternative priors and basis sets are objectively compared via evidence, and Bayes automatically implements Occam’s razor, yielding an elegant interpretation of the effective number of parameters determined by the data.

Abstract

Although Bayesian analysis has been in use since Laplace, the Bayesian method of model-comparison has only recently been developed in depth. In this paper, the Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data. The concepts and methods described are quite general and can be applied to many other data modeling problems. Regularizing constants are set by examining their posterior probability distribution. Alternative regularizers (priors) and alternative basis sets are objectively compared by evaluating the evidence for them. “Occam's razor” is automatically embodied by this process. The way in which Bayes infers the values of regularizing constants and noise levels has an elegant interpretation in terms of the effective number of parameters determined by the data set. This framework is due to Gull and Skilling.