Communications in Statistics - Simulation and Computation · 2007 · 91 citations · 22 references
ComputingComputer ScienceAlgorithmsMachine LearningTime Series AnalysisBayesian Nonparametric MethodsGaussian ProcessesMatrix ComputationMaximum Likelihood EstimationParameter EstimationSystem IdentificationAccuracyBayesian RegressionCovariance Matrix ApproximationCovariance Matrix EstimationGaussian Process RegressionHammerstein Model IdentificationHammerstein-wiener ModelsNonparametric Regression
Gaussian process (GP) is a Bayesian nonparametric regression model, showing good performance in various applications. However, during its model-tuning procedure, the GP implementation suffers from numerous covariance-matrix inversions of expensive O(N3) operations, where N is the matrix dimension. In this article, we propose using the quasi-Newton BFGS O(N2)-operation formula to approximate/replace recursively the inverse of covariance matrix at every iteration. The implementation accuracy is guaranteed carefully by a matrix-trace criterion and by the restarts technique to generate good initial guesses. A number of numerical tests are then performed based on the sinusoidal regression example and the Wiener–Hammerstein identification example. It is shown that by using the proposed implementation, more than 80% O(N3) operations could be eliminated, and a typical speedup of 5–9 could be achieved as compared to the standard maximum-likelihood-estimation (MLE) implementation commonly used in Gaussian process regression.
22
Numerical recipes in Pascal: the art of scientific computing
Choice Reviews Online · 1990 · 11.9K citations
Noel Cressie · Terra Nova · 1992 · 8.9K citations
Computing, Geography, Geoinformatics +12
Sparse On-Line Gaussian Processes
Lehel Csató, Manfred Opper · Neural Computation · 2002 · 763 citations · Full text