Sparse Multivariate Regression With Covariance Estimation

TLDR

The authors propose a sparse multivariate regression estimator that accounts for correlated responses. They develop MRCE, a penalized‑likelihood method that jointly estimates regression coefficients and covariance, and provide efficient optimization and fast approximation algorithms, demonstrated on a finance asset‑return prediction example. Simulation studies show MRCE outperforms competitors on highly correlated responses, and an R package with data and code is available online.

Abstract

We propose a procedure for constructing a sparse estimator of a multivariate regression coefficient matrix that accounts for correlation of the response variables. This method, which we call multivariate regression with covariance estimation (MRCE), involves penalized likelihood with simultaneous estimation of the regression coefficients and the covariance structure. An efficient optimization algorithm and a fast approximation are developed for computing MRCE. Using simulation studies, we show that the proposed method outperforms relevant competitors when the responses are highly correlated. We also apply the new method to a finance example on predicting asset returns. An R-package containing this dataset and code for computing MRCE and its approximation are available online.