SIAM Journal on Optimization · 2013 · 302 citations · 35 references
ComputingComputer ScienceAlgorithmsOptimizationScientific ComputingGlobal OptimizationGradient-based OptimizationIterative SolversNonlinear ProgrammingNumerical AlgorithmsUnconstrained OptimizationAdaptive OptimizationConvergenceGlobal Convergence AnalysisGradient DescentGradient Method OptimizationNonlinear Conjugate GradientOptimization AlgorithmsSteepest Descent
In this paper, we seek the conjugate gradient direction closest to the direction of the scaled memoryless BFGS method and propose a family of conjugate gradient methods for unconstrained optimization. An improved Wolfe line search is also proposed, which can avoid a numerical drawback of the original Wolfe line search and guarantee the global convergence of the conjugate gradient method under mild conditions. To accelerate the algorithm, we introduce adaptive restarts along negative gradients based on the extent to which the function approximates some quadratic function during previous iterations. Numerical experiments with the CUTEr collection show that the proposed algorithm is promising.
35
Function minimization by conjugate gradients
R. Fletcher · The Computer Journal · 1964 · 4.9K citations · Full text
Restart procedures for the conjugate gradient method
M. J. D. Powell · Mathematical Programming · 1977 · 1.7K citations