Joint estimation of multiple graphical models

TLDR

Gaussian graphical models estimate inverse covariance matrices to capture dependence among variables, but fitting a single model masks heterogeneity while separate models ignore shared structure. The paper develops an estimator that jointly learns graphical models across categories, preserving shared structure while allowing category‑specific differences. The estimator uses a hierarchical penalty to remove common zeros across inverse covariance matrices, and is shown to be asymptotically consistent and sparse in high dimensions, with performance illustrated on simulated networks and a semantic network application. The estimator is asymptotically consistent and sparse in high dimensions, performs well on simulated networks, and successfully learns semantic connections among terms from computer science webpages.

Abstract

Gaussian graphical models explore dependence relationships between random variables, through the estimation of the corresponding inverse covariance matrices. In this paper we develop an estimator for such models appropriate for data from several graphical models that share the same variables and some of the dependence structure. In this setting, estimating a single graphical model would mask the underlying heterogeneity, while estimating separate models for each category does not take advantage of the common structure. We propose a method that jointly estimates the graphical models corresponding to the different categories present in the data, aiming to preserve the common structure, while allowing for differences between the categories. This is achieved through a hierarchical penalty that targets the removal of common zeros in the inverse covariance matrices across categories. We establish the asymptotic consistency and sparsity of the proposed estimator in the high-dimensional case, and illustrate its performance on a number of simulated networks. An application to learning semantic connections between terms from webpages collected from computer science departments is included.