Optimal Mean-Squared-Error Batch Sizes

Abstract

When an estimator of the variance of the sample mean is parameterized by batch size, one approach for selecting batch size is to pursue the minimal mean squared error (mse). We show that the convergence rate of the variance of the sample mean, and the bias of estimators of the variance of the sample mean, asymptotically depend on the data process only through its marginal variance and the sum of the autocorrelations weighted by their absolute lags. Combining these results with variance results of Goldsman and Meketon, we obtain explicit asymptotic approximations for mse, optimal batch size, optimal mse, and robustness for four quadratic-form estimators of the variance of the sample mean. Our empirical results indicate that the asymptotic approximations are reasonably accurate for sample sizes seen in practice. Although we do not discuss batch-size estimation procedures, the empirical results suggest that the explicit asymptotic batch-size approximation, which depends only on a summary measure (which we refer to as the balance point) of the nonnegative-lag autocorrelations, is a reasonable foundation for such procedures.