Towards a Black Box Algorithm for Nonlinear Function Approximation over High‐Dimensional Domains

Abstract

We propose an algorithm for constructing nonlinear models from high‐dimensional scattered data. The algorithm progresses iteratively, adding a new function at each step to refine the model. The placement of the functions is driven by a statistical hypothesis test that reveals geometric structure when it fails. At each step the added function is fit to data contained in a spatio‐temporally defined local region to determine the parameters, particularly the scale of the local model. Unlike currently available techniques for nonlinear function fitting over scattered data, the proposed method requires no ad hoc parameters. Thus, the number of basis functions required for an accurate fit is determined automatically by the algorithm. We illustrate the approach using several illustrative problems including modeling data on manifolds and the prediction of financial time‐series. The algorithm is presented in the context of radial basis functions but in principle can be employed with other methods for function approximation, such as multilayer perceptrons.