Fast and robust fixed-point algorithms for independent component analysis

TLDR

Independent component analysis (ICA) transforms an observed multidimensional random vector into components that are statistically as independent as possible. The study introduces new contrast functions for ICA based on maximum‑entropy approximations and proposes simple fixed‑point algorithms to optimize them. The authors combine Comon's information‑theoretic and projection‑pursuit approaches, using maximum‑entropy contrast functions that enable full decomposition via mutual‑information minimization and component‑wise projection pursuit, and analyze their statistical properties under a linear mixture model to guide robust or minimum‑variance choices. The analysis demonstrates how to select contrast functions that are robust and/or achieve minimum variance under the linear mixture model.

Abstract

Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. We use a combination of two different approaches for linear ICA: Comon's information theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple fixed-point algorithms for practical optimization of the contrast functions.