Regularization Paths for Generalized Linear Models via Coordinate Descent

TLDR

The study focuses on generalized linear models such as linear, two‑class logistic, and multinomial regression with convex penalties including lasso, ridge, and elastic net. The authors aim to develop fast algorithms for estimating these generalized linear models with convex penalties. They employ cyclical coordinate descent along a regularization path, enabling efficient handling of large problems and sparse features. Timing comparisons show the new algorithms are considerably faster than existing methods.

Abstract

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multi- nomial regression problems while the penalties include ℓ₁ (the lasso), ℓ₂ (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.