IEEE Transactions on Neural Networks · 1993 · 143 citations · 40 references
ComputingComputer ScienceAlgorithmsMachine LearningOptimizationBias-variance TradeoffNeural NetworksOptimization For Machine LearningPattern RecognitionRepresentation LearningSupervised LearningLeast Squares ProblemsPretrainingSelection Criterion DesignTraining
For clean data with deterministic relationships, concise training sets that minimize integrated squared bias are desired. The study aims to reduce data requirements for learning by deriving a method to select exemplars for training a multilayer feedforward network on clean deterministic data. The authors sequentially choose training examples that maximize the reduction in network squared error, using a criterion derived from integrated squared bias (DISB) that aligns with least‑squares learning. Experimental results and illustrations show that selecting exemplars with the proposed criterion reduces computation while maintaining performance during network training.
The authors derive a method for selecting exemplars for training a multilayer feedforward network architecture to estimate an unknown (deterministic) mapping from clean data, i.e., data measured either without error or with negligible error. The objective is to minimize the data requirement of learning. The authors choose a criterion for selecting training examples that works well in conjunction with the criterion used for learning, here, least squares. They proceed sequentially, selecting an example that, when added to the previous set of training examples and learned, maximizes the decrement of network squared error over the input space. When dealing with clean data and deterministic relationships, concise training sets that minimize the integrated squared bias (ISB) are desired. The ISB is used to derive a selection criterion for evaluating individual training examples, the DISB, that is maximized to select new exemplars. They conclude with graphical illustrations of the method, and demonstrate its use during network training. Experimental results indicate that training upon exemplars selected in this fashion can save computation in general purpose use as well.
40
Multilayer feedforward networks are universal approximators
Kurt Hornik, Maxwell B. Stinchcombe, Halbert White · Neural Networks · 1989 · 21.2K citations
William H. Press, Saul A. Teukolsky, William T. Vetterling et al. · 1994 · 15.4K citations
Savitzky-Golay Smoothing Filters
William H. Press, Saul A. Teukolsky · Computers in Physics · 1990 · 11.8K citations · Full text
PE, P. Billingsley · Journal of the American Statistical Association · 1996 · 6.7K citations
Empirical Model Building and Response Surfaces
Anne E. Freeny, George E. P. Box, Norman R. Draper · Technometrics · 1988 · 5.1K citations