2002 · 16 citations · 23 references
Natural SciencesMathematicsDynamical SystemsMachine LearningOptimizationControl Of Dynamical SystemsDynamic OptimizationDynamic SimulationDynamic Systems AnalysisNeural NetworksTrajectory AnalysisAdjoint FormulationGradient DescentGradient Method Optimization
The application of artificial neural networks to dynamical systems has been constrained by the non-dynamical nature of popular network architectures. Many of the difficulties that ensue-large network sizes, long training times, the need to predetermine buffer lengths-can be overcome with dynamic neural networks. The minimization of a quadratic performance index is considered for trajectory tracking or process simulation applications. Two approaches for gradient computation are discussed: forward and adjoint sensitivity analysis. The computational complexity of the latter is significantly less, but it requires a backward integration capability. We also discuss two parameter updating methods: the gradient descent method and Levenberg-Marquardt approach.
23
Learning representations by back-propagating errors
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams · Nature · 1986 · 30.9K citations
Jeffrey L. Elman · Cognitive Science · 1990 · 10.8K citations