Expert Systems with Applications · 2013 · 41 citations · 51 references
ComputingComputer ScienceForecastingMachine LearningOptimizationComputational Time Series AnalysisForecast EvaluationNeural NetworksTime Series ForecastingTime Series ModelingTime Series PredictionActivation FunctionsAsymmetric ActivationNeural ForecastingNeural Network ModelingNeural Network PredictionNeural Time Series ForecastingOptimal Weight Selection
The use of neural network models for time series forecasting has been motivated by experimental results that indicate high capacity for function approximation with good accuracy. Generally, these models use activation functions with fixed parameters. However, it is known that the choice of activation function strongly influences the complexity and neural network performance and that a limited number of activation functions has been used in general. We describe the use of an asymmetric activation functions family with free parameter for neural networks. We prove that the activation functions family defined, satisfies the requirements of the universal approximation theorem We present a methodology for global optimization of the activation functions family with free parameter and the connections between the processing units of the neural network. The main idea is to optimize, simultaneously, the weights and activation function used in a Multilayer Perceptron (MLP), through an approach that combines the advantages of simulated annealing, tabu search and a local learning algorithm. We have chosen two local learning algorithms: the backpropagation with momentum (BPM) and Levenberg–Marquardt (LM). The overall purpose is to improve performance in time series forecasting.
51
Genetic algorithms in search, optimization, and machine learning
Choice Reviews Online · 1989 · 49.3K citations
Optimization by Simulated Annealing
Scott Kirkpatrick, C. D. Gelatt, M.P. Vecchi · Science · 1983 · 44.6K citations
Equation of State Calculations by Fast Computing Machines
N. Metropolis, Arianna W. Rosenbluth, M. N. Rosenbluth et al. · The Journal of Chemical Physics · 1953 · 37.1K citations
Multilayer feedforward networks are universal approximators
Kurt Hornik, Maxwell B. Stinchcombe, Halbert White · Neural Networks · 1989 · 21.2K citations