On the Hyperprior Choice for the Global Shrinkage Parameter in the\n Horseshoe Prior

Juho Piironen, Aki Vehtari

arXiv (Cornell University) · 2016 · 66 citations · 0 references

DOIFull text

Open access

Concepts

Abstract

The horseshoe prior has proven to be a noteworthy alternative for sparse\nBayesian estimation, but as shown in this paper, the results can be sensitive\nto the prior choice for the global shrinkage hyperparameter. We argue that the\nprevious default choices are dubious due to their tendency to favor solutions\nwith more unshrunk coefficients than we typically expect a priori. This can\nlead to bad results if this parameter is not strongly identified by data. We\nderive the relationship between the global parameter and the effective number\nof nonzeros in the coefficient vector, and show an easy and intuitive way of\nsetting up the prior for the global parameter based on our prior beliefs about\nthe number of nonzero coefficients in the model. The results on real world data\nshow that one can benefit greatly -- in terms of improved parameter estimates,\nprediction accuracy, and reduced computation time -- from transforming even a\ncrude guess for the number of nonzero coefficients into the prior for the\nglobal parameter using our framework.\n