Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning

TLDR

The paper reviews the majorization‑minimization framework, offering guidance for designing low‑cost, problem‑driven algorithms. It presents MM’s fundamentals, convergence guarantees, extensions, acceleration techniques, and surrogate‑function construction, then illustrates its application across signal processing, communications, and machine learning. The framework is successfully applied to numerous problems in signal processing, communications, and machine learning.

Abstract

This paper gives an overview of the majorization-minimization (MM) algorithmic framework, which can provide guidance in deriving problem-driven algorithms with low computational cost. A general introduction of MM is presented, including a description of the basic principle and its convergence results. The extensions, acceleration schemes, and connection to other algorithmic frameworks are also covered. To bridge the gap between theory and practice, upperbounds for a large number of basic functions, derived based on the Taylor expansion, convexity, and special inequalities, are provided as ingredients for constructing surrogate functions. With the pre-requisites established, the way of applying MM to solving specific problems is elaborated by a wide range of applications in signal processing, communications, and machine learning.