Shrinkage Algorithms for MMSE Covariance Estimation

TLDR

The paper addresses MMSE covariance estimation for Gaussian samples. The study aims to improve the Ledoit‑Wolf method by conditioning on a sufficient statistic and to propose an iterative approach approximating the clairvoyant shrinkage estimator. The authors develop two shrinkage estimators—RBLW, derived via Rao‑Blackwellization of the Ledoit‑Wolf estimator, and OAS, an iterative method converging to a closed‑form oracle‑approximating shrinkage estimator—suitable for high‑dimensional, small‑sample Gaussian data. RBLW provably dominates the Ledoit‑Wolf estimator, while OAS can outperform RBLW in high‑dimensional, small‑sample regimes, and both estimators are simple, implementable, and effective for adaptive beamforming.

Abstract

We address covariance estimation in the sense of minimum mean-squared error (MMSE) for Gaussian samples. Specifically, we consider shrinkage methods which are suitable for high dimensional problems with a small number of samples (large p small n). First, we improve on the Ledoit-Wolf (LW) method by conditioning on a sufficient statistic. By the Rao-Blackwell theorem, this yields a new estimator called RBLW, whose mean-squared error dominates that of LW for Gaussian variables. Second, to further reduce the estimation error, we propose an iterative approach which approximates the clairvoyant shrinkage estimator. Convergence of this iterative method is established and a closed form expression for the limit is determined, which is referred to as the oracle approximating shrinkage (OAS) estimator. Both RBLW and OAS estimators have simple expressions and are easily implemented. Although the two methods are developed from different persepctives, their structure is identical up to specified constants. The RBLW estimator provably dominates the LW method. Numerical simulations demonstrate that the OAS approach can perform even better than RBLW, especially when n is much less than p. We also demonstrate the performance of these techniques in the context of adaptive beamforming.