Rates of Convergence of the Functional $k$-Nearest Neighbor Estimate

Abstract

Let F be a separable Banach space, and let (X, Y) be a random pair taking values in F × R. Motivated by a broad range of potential applications, we investigate rates of convergence of the k-nearest neighbor estimate r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> (x) of the regression function r(x) = E[Y|X = x], based on n independent copies of the pair (X, Y). Using compact embedding theory, we present explicit and general finite sample bounds on the expected squared difference E[r <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> (X) - r(X)] <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , and particularize our results to classical function spaces such as Sobolev spaces, Besov spaces, and reproducing kernel Hilbert spaces.