Projected Hessian Updating Algorithms for Nonlinearly Constrained Optimization

Abstract

We consider the problem of minimizing a smooth function of n variables subject to m smooth equality constraints. We begin by describing various approaches to Newton’s method for this problem, with emphasis on the recent work of Goodman. This leads to the proposal of a Broyden-type method which updates an $n \times (n - m)$ matrix approximating a “one-sided projected Hessian” of a Lagrangian function. This method is shown to converge Q-superlinearly. We also give a new short proof of the Boggs-Tolle-Wang necessary and sufficient condition for Q-superlinear convergence of a class of quasi-Newton methods for solving this problem. Finally, we describe an algorithm which updates an approximation to a “two-sided projected Hessian,” a symmetric matrix of order $n - m$ which is generally positive definite near a solution. We present several new variants of this algorithm and show that under certain conditions they all have a local two-step Q-superlinear convergence property, even though only one set of gradients is evaluated per iteration. Numerical results are presented, indicating that the methods may be very useful in practice.