Structure Discovery in Nonparametric Regression through Compositional Kernel Search

Abstract

Despite its importance, choosing the struc-tural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of struc-tures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable com-ponents and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used ker-nels and kernel combination methods on a variety of prediction tasks. 1.