Neural Computation · 1992 · 3.5K citations · 83 references
Artificial IntelligenceEngineeringMachine LearningAi FoundationFeedforward Neural NetworksRecurrent Neural NetworkData SciencePattern RecognitionSparse Neural NetworkSupervised LearningCognitive ScienceMachine VisionComputational Learning TheoryMachine Learning ModelComputer ScienceNeural NetworksDeep LearningMedical Image ComputingNonparametric InferenceEvolving Neural NetworkNonparametric Regression Estimators
Feedforward neural networks trained by backpropagation serve as nonparametric regression estimators. The paper offers a tutorial linking nonparametric inference to neural networks, using a statistical perspective to expose their strengths and weaknesses. The authors illustrate concepts with recognition experiments on artificial data and handwritten numerals. They conclude that modern feedforward neural networks are inadequate for challenging perception and learning tasks, and that representation—not learning—is the core issue, as supported by handwritten numeral experiments.
Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.
83
Learning representations by back-propagating errors
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams · Nature · 1986 · 29.7K citations
Classification and Regression Trees.
Alexander Gordon, Leo Breiman, Jerome H. Friedman et al. · Biometrics · 1984 · 23.8K citations
Classification and Regression Trees.
John Van Ryzin, Leo Breiman, Jerome H. Friedman et al. · Journal of the American Statistical Association · 1986 · 21K citations