SIAM Journal on Optimization · 1998 · 484 citations · 8 references
ComputingComputer ScienceComputational GeometryOptimizationScientific ComputingConvergence AnalysisIterative SolversNumerical AlgorithmsNumerical AnalysisNumerical ConvergenceNumerical StabilityConvergenceDescentGradient DescentNonstationary Point ProcessesPiecewise Continuous FunctionsSteepest Descent
The study investigates how the Nelder–Mead simplex method can converge to a nonstationary point on a specific family of two‑variable continuous functions. The authors examine a family of two‑variable continuous, strictly convex functions with up to three derivatives, where the Nelder–Mead algorithm repeatedly performs inside contraction with the best vertex fixed, and analyze the stability of this behavior. The simplices collapse to a straight line orthogonal to the steepest descent direction, a behavior that cannot occur for functions with more than three continuous derivatives.
This paper analyzes the behavior of the Nelder--Mead simplex method for a family of examples which cause the method to converge to a nonstationary point. All the examples use continuous functions of two variables. The family of functions contains strictly convex functions with up to three continuous derivatives. In all the examples the method repeatedly applies the inside contraction step with the best vertex remaining fixed. The simplices tend to a straight line which is orthogonal to the steepest descent direction. It is shown that this behavior cannot occur for functions with more than three continuous derivatives. The stability of the examples is analyzed.
8
A Simplex Method for Function Minimization
J. A. Nelder, R. Mead · The Computer Journal · 1965 · 28.9K citations
On the Convergence of Pattern Search Algorithms
Virginia Torczon · SIAM Journal on Optimization · 1997 · 1.3K citations