Kullback-Leibler information measure for studying convergence rates of densities and distributions

Abstract

The Kullback-Leibler (KL) information measure l(f/sub 1/:f/sub 2/) is proposed as an index for studying rates of convergence of densities and distribution functions. To this end, upper bounds in terms of l(f/sub 1/:f/sub 2/) for several distance functions for densities and for distribution functions are obtained. Many illustrations of the use of this technique are given. It is shown, for example, that the sequence of KL information measures converges to zero more slowly for a normalized sequence of gamma random variables converging to its limiting normal distribution than for a normalized sequence of largest order statistics from an exponential distribution converging to its limiting extreme value distribution. Furthermore, a sequence of KL information measures for log-normal random variables approaching normality converges more slowly to zero than for a sequence of normalized gamma random variables.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>