Simple mixture model for sparse overcomplete ICA

Abstract

The use of mixture of Gaussians (MoGs) for noisy and overcomplete independent component analysis (ICA) when the source distributions are very sparse is explored. The sparsity model can often be justified if an appropriate transform, such as the modified discrete cosine transform, is used. Given the sparsity assumption, a number of simplifying approximations are introduced to the observation density that avoid the exponential growth of mixture components. An efficient clustering algorithm is derived whose complexity grows linearly with the number of sources and it is shown that it is capable of performing reasonable separation.