Stability Approach to Regularization Selection (StARS) for High\n Dimensional Graphical Models

Abstract

A challenging problem in estimating high-dimensional graphical models is to\nchoose the regularization parameter in a data-dependent way. The standard\ntechniques include $K$-fold cross-validation ($K$-CV), Akaike information\ncriterion (AIC), and Bayesian information criterion (BIC). Though these methods\nwork well for low-dimensional problems, they are not suitable in high\ndimensional settings. In this paper, we present StARS: a new stability-based\nmethod for choosing the regularization parameter in high dimensional inference\nfor undirected graphs. The method has a clear interpretation: we use the least\namount of regularization that simultaneously makes a graph sparse and\nreplicable under random sampling. This interpretation requires essentially no\nconditions. Under mild conditions, we show that StARS is partially sparsistent\nin terms of graph estimation: i.e. with high probability, all the true edges\nwill be included in the selected model even when the graph size diverges with\nthe sample size. Empirically, the performance of StARS is compared with the\nstate-of-the-art model selection procedures, including $K$-CV, AIC, and BIC, on\nboth synthetic data and a real microarray dataset. StARS outperforms all these\ncompeting procedures.\n