The Bayesian Lasso

TLDR

The Lasso estimate can be viewed as a Bayesian posterior mode when regression coefficients have independent Laplace priors. The Bayesian Lasso employs a hierarchical model with normal priors and exponential variance priors, enabling Gibbs sampling and tractable full conditionals via an inverse‑Gaussian link, and allows both Bayesian and likelihood selection of the Lasso parameter while extending to other Lasso‑related estimators. The Bayesian Lasso yields credible intervals that aid variable selection.

Abstract

The Lasso estimate for linear regression parameters can be interpreted as a Bayesian posterior mode estimate when the regression parameters have independent Laplace (i.e., double-exponential) priors. Gibbs sampling from this posterior is possible using an expanded hierarchy with conjugate normal priors for the regression parameters and independent exponential priors on their variances. A connection with the inverse-Gaussian distribution provides tractable full conditional distributions. The Bayesian Lasso provides interval estimates (Bayesian credible intervals) that can guide variable selection. Moreover, the structure of the hierarchical model provides both Bayesian and likelihood methods for selecting the Lasso parameter. Slight modifications lead to Bayesian versions of other Lasso-related estimation methods, including bridge regression and a robust variant.