Bootstrap Methods: Another Look at the Jackknife

TLDR

Given a random sample from an unknown distribution, the problem is to estimate the sampling distribution of a specified statistic from the observed data, and standard jackknife theory provides approximate mean and variance for parameter differences. The paper introduces the bootstrap as a general method that works satisfactorily on various estimation problems. The authors illustrate the bootstrap by applying it to examples such as the variance of the sample median, error rates in linear discriminant analysis, ratio estimation, and regression parameter estimation. The jackknife is demonstrated to be a linear approximation of the bootstrap.

Abstract

We discuss the following problem: given a random sample $\mathbf{X} = (X_1, X_2, \cdots, X_n)$ from an unknown probability distribution $F$, estimate the sampling distribution of some prespecified random variable $R(\mathbf{X}, F)$, on the basis of the observed data $\mathbf{x}$. (Standard jackknife theory gives an approximate mean and variance in the case $R(\mathbf{X}, F) = \theta(\hat{F}) - \theta(F), \theta$ some parameter of interest.) A general method, called the "bootstrap," is introduced, and shown to work satisfactorily on a variety of estimation problems. The jackknife is shown to be a linear approximation method for the bootstrap. The exposition proceeds by a series of examples: variance of the sample median, error rates in a linear discriminant analysis, ratio estimation, estimating regression parameters, etc.