Genetic Programming and Evolvable Machines · 2019 · 110 citations · 37 references
ComputingComputer ScienceAlgorithmsMachine LearningOptimizationEvolutionary ComputationGenetic AlgorithmsGenetic ProgrammingGradient-based OptimizationOptimization For Machine LearningParameter OptimizationAutomatic DifferentiationDifferentiationLocal SearchLocal Search AlgorithmsNonlinear Least SquaresParameter IdentificationSymbolic RegressionTree-based Genetic Programming
Abstract In this paper we analyze the effects of using nonlinear least squares for parameter identification of symbolic regression models and integrate it as local search mechanism in tree-based genetic programming. We employ the Levenberg–Marquardt algorithm for parameter optimization and calculate gradients via automatic differentiation. We provide examples where the parameter identification succeeds and fails and highlight its computational overhead. Using an extensive suite of symbolic regression benchmark problems we demonstrate the increased performance when incorporating nonlinear least squares within genetic programming. Our results are compared with recently published results obtained by several genetic programming variants and state of the art machine learning algorithms. Genetic programming with nonlinear least squares performs among the best on the defined benchmark suite and the local search can be easily integrated in different genetic programming algorithms as long as only differentiable functions are used within the models.
37
Genetic algorithms in search, optimization, and machine learning
Choice Reviews Online · 1989 · 49.3K citations
Optimization by Simulated Annealing
Scott Kirkpatrick, C. D. Gelatt, M.P. Vecchi · Science · 1983 · 44.6K citations