Quasi-Newton Methods and their Application to Function Minimisation

Abstract

Introduction.The solution of a set of n nonlinear simultaneous equations, which may be writtencan in general only be found by an iterative process in which successively better, in some sense, approximations to the solution are computed.Of the methods available most rely on evaluating at each stage of the calculation a set of residuals and from these obtaining a correction to each element of the approximate solution.The most common way of doing this is to take each correction to be a suitable linear combination of the residuals.There is, of course, no reason in principle why more elaborate schemes should not be used but they are difficult both to analyse theoretically and to implement in practice.The minimisation of a function of n variables, for which it is possible to obtain analytic expressions for the n first partial derivatives, is a particular example of this type of problem.Any technique used to solve nonlinear equations may be applied to the expressions for the partial derivatives but, because it is known in this case that the residuals form the gradient of some function, it is possible to introduce refinements into the method of solution to take account of this extra information.Since, in addition, the value of the function itself is known, further refinements are possible.