Optimization of k nearest neighbor density estimates

Abstract

Nonparametric density estimation using the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> -nearest-neighbor approach is discussed. By developing a relation between the volume and the coverage of a region, a functional form for the optimum <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> in terms of the sample size, the dimensionality of the observation space, and the underlying probability distribution is obtained. Within the class of density functions that can be made circularly symmetric by a linear transformation, the optimum matrix for use in a quadratic form metric is obtained. For Gaussian densities this becomes the inverse covariance matrix that is often used without proof of optimality. The close relationship of this approach to that of Parzen estimators is then investigated.