A Sparse-Group Lasso

TLDR

For high‑dimensional supervised learning, exploiting problem‑specific assumptions can improve accuracy. The study introduces a sparse‑group lasso model for linear regression that applies ℓ1 and ℓ2 penalties to grouped covariates presumed to have sparse effects at both group and within‑group levels. An accelerated generalized gradient descent algorithm is proposed to fit the model, and the method is extended to convex loss functions. The model achieves group‑wise and within‑group sparsity, and simulations demonstrate its efficacy and algorithmic efficiency. Online supplementary material is available.

Abstract

For high-dimensional supervised learning problems, often using problem-specific assumptions can lead to greater accuracy. For problems with grouped covariates, which are believed to have sparse effects both on a group and within group level, we introduce a regularized model for linear regression with ℓ1 and ℓ2 penalties. We discuss the sparsity and other regularization properties of the optimal fit for this model, and show that it has the desired effect of group-wise and within group sparsity. We propose an algorithm to fit the model via accelerated generalized gradient descent, and extend this model and algorithm to convex loss functions. We also demonstrate the efficacy of our model and the efficiency of our algorithm on simulated data. This article has online supplementary material.