Algorithms for Non-negative Matrix Factorization

TLDR

NMF is a useful decomposition technique for multivariate data. The study analyzes two multiplicative NMF algorithms that differ only slightly in their update factors, proves their monotonic convergence using an auxiliary function, and interprets them as diagonally rescaled gradient descent with optimally chosen rescaling. One algorithm minimizes least‑squares error, whereas the other minimizes generalized Kullback‑Leibler divergence.

Abstract

Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition for multivariate data. Two different multiplicative algorithms for NMF are analyzed. They differ only slightly in the multiplicative factor used in the update rules. One algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence. The monotonic convergence of both algorithms can be proven using an auxiliary function analogous to that used for proving convergence of the Expectation-Maximization algorithm. The algorithms can also be interpreted as diagonally rescaled gradient descent, where the rescaling factor is optimally chosen to ensure convergence.