Combinatorial selection and least absolute shrinkage via the Clash algorithm

Abstract

The least absolute shrinkage and selection operator (LASSO) for linear regression exploits the geometric interplay of the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -data error objective and the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm constraint to arbitrarily select sparse models. Guiding this uninformed selection process with sparsity models has been precisely the center of attention over the last decade in order to improve learning performance. To this end, we alter the selection process of LASSO to explicitly leverage combinatorial sparsity models (CSMs) via the combinatorial selection and least absolute shrinkage (Clash) operator. We provide concrete guidelines how to leverage combinatorial constraints within Clash, and characterize CLASH's guarantees as a function of the set restricted isometry constants of the sensing matrix. Finally, our experimental results show that Clash can outperform both LASSO and model-based compressive sensing in sparse estimation.