Learning Sparse Representation Using Iterative Subspace Identification

Abstract

In this paper, we introduce the iterative subspace identification (ISI) algorithm for learning subspaces in which the data may live. Our subspace identification method differs from currently available method in its ability to infer the dimension of the subspaces from the data without prior knowledge. The learned subspaces can be combined to produce a data driven overcomplete dictionary with good sparseness and generalizability qualities, or can be directly exploited in applications where block sparseness is needed. We describe the ISI algorithm and a complementary optimization method. We demonstrate the ability of the proposed method to produce sparse representations comparable to those achieved with the K-SVD algorithm, but with less than one eighth the training time. Furthermore, the computation savings allows us to develop a shift-tolerant training procedure. We also illustrate its benefits in underdetermined blind source separation of audio, where performance is directly impacted by the sparseness of the representation.