Technometrics · 2019 · 30 citations · 29 references
ArchitectureDesignDesign OptimizationExperimental DesignOptimizationComputational ModelingDesign Space ExplorationGaussian Process ModelingGaussian ProcessesSurrogate ModelingSurrogate-based OptimizationExperimental Study DesignGaussian Process SurrogatesLatin Hypercube SamplingSample Size Estimation
Abstract A common challenge in computer experiments and related fields is to efficiently explore the input space using a small number of samples, that is, the experimental design problem. Much of the recent focus in the computer experiment literature, where modeling is often via Gaussian process (GP) surrogates, has been on space-filling designs, via maximin distance, Latin hypercube, etc. However, it is easy to demonstrate empirically that such designs disappoint when the model hyperparameterization is unknown, and must be estimated from data observed at the chosen design sites. This is true even when the performance metric is prediction-based, or when the target of interest is inherently or eventually sequential in nature, such as in blackbox (Bayesian) optimization. Here we expose such inefficiencies, showing that in many cases a purely random design is superior to higher-powered alternatives. We then propose a family of new schemes by reverse engineering the qualities of the random designs which give the best estimates of GP length scales. Specifically, we study the distribution of pairwise distances between design elements, and develop a numerical scheme to optimize those distances for a given sample size and dimension. We illustrate how our distance-based designs, and their hybrids with more conventional space-filling schemes, outperform in both static (one-shot design) and sequential settings.
29
R: A Language and Environment for Statistical Computing
R Core Team · 2000 · 353K citations · Full text
Natural Sciences, Computer Science, Statistical Computing +10
Design and Analysis of Computer Experiments
Jerome Sacks, William J. Welch, Toby J. Mitchell et al. · Statistical Science · 1989 · 7K citations · Full text
A Limited Memory Algorithm for Bound Constrained Optimization
Richard H. Byrd, Peihuang Lu, Jorge Nocedal et al. · SIAM Journal on Scientific Computing · 1995 · 6.2K citations
G. Matheron · Economic Geology · 1963 · 4.8K citations