1994 · 136 citations · 12 references
ComputingComputer EngineeringComputer ScienceAlgorithmsMachine LearningOptimizationLoss FunctionNeural NetworksOptimization For Machine LearningPattern RecognitionBackpropagationFeedforward Neural NetworksMultilayer PerceptronOptimization AlgorithmsPerceptron
this report also have been published on ESANN '93 [Schiffmann et al., 1993]. The dataset used in this comparision is available by anonymous ftp (FTP server: ics.uci.edu, files: pub/machine-learningdatabases /thyroid-disease/ann). Backpropagation is one of the most popular training algorithms for multilayer perceptrons. Unfortunately it can be very slow for practical applications. Over the last years many improvement strategies have been developed to speed up backpropagation. It's very difficult to compare these different techniques, because most of them have been tested on very special data sets. The reported results are based on some kind of tiny and artificial training sets like XOR, encoder or decoder. It's very doubtful if this results hold for a much more complicate practical application. In these report an overview of many different speedup techniques is given. All of them are tested on a very hard practical classification task, which consists of a big medical data set. As you will see many of these optimized algorithms fail in learning the data set. 2 Application
12
Introduction to the theory of neural computation
Neural Networks · 1994 · 6.5K citations
Increased rates of convergence through learning rate adaptation
Robert A. Jacobs · Neural Networks · 1988 · 1.8K citations