A novel measure for independent component analysis (ICA)

Abstract

Measures of independence (and dependence) are fundamental in many areas of engineering and signal processing. Shannon introduced the idea of information entropy which has a sound theoretical foundation but sometimes is not easy to implement in engineering applications. In this paper, Renyi's entropy is used and a novel independence measure is proposed. When integrated with a nonparametric estimator of the probability density function (Parzen Window), the measure can be related to the "potential energy of the samples" which is easy to understand and implement. The experimental results on blind source separation confirm the theory. Although the work is preliminary, the "potential energy" method is rather general and will have many applications.