Updating quasi-Newton matrices with limited storage

TLDR

The paper proposes a storage‑efficient BFGS preconditioning approach that updates quasi‑Newton matrices using only the most recent m iterations. The authors derive an update rule that replaces the oldest curvature pair with the newest at each iteration, yielding a limited‑storage quasi‑Newton matrix that is evaluated numerically against standard algorithms. The limited‑storage matrices exhibit desirable theoretical properties and perform competitively with established methods in numerical tests.

Abstract

We study how to use the BFGS quasi-Newton matrices to precondition minimization methods for problems where the storage is critical. We give an update formula which generates matrices using information from the last <italic>m</italic> iterations, where <italic>m</italic> is any number supplied by the user. The quasi-Newton matrix is updated at every iteration by dropping the oldest information and replacing it by the newest information. It is shown that the matrices generated have some desirable properties. The resulting algorithms are tested numerically and compared with several well-known methods.