Novel interpretation for Levenberg-Marquardt algorithm

Abstract

The Levenberg-Marquard(tLM) algorithm is closely related with the Least Squares(LS) approach.The Scaled Total Least Squares(STLS) approach is a unification and generalization of the LS,Data Least Squares(DLS) and Total Least Squares (TLS) approaches,but its relation with the LM algorithm is not clear.In this paper,a STLS algorithm and its interpretations via subspace and topology are proposed.The relation of the STLS approach and the LM algorithm are explored by matrix decomposition and the results show that:LS solutions are converted to STLS solutions when the damp factors are introduced.The robustness and convergence performance of the LM algorithm are reached by eliminating the noise subspace and controlling the condition number of the coefficient matrix.The capabilities in solving the over-parameterized problems of the LM algorithm are determined by the robustness of the STLS approach.