Kernel Methods for Measuring Independence

Abstract

We introduce two new functionals, the constrained covariance and the kernel mutual information,
\nto measure the degree of independence of random variables. These quantities are both based on
\nthe covariance between functions of the random variables in reproducing kernel Hilbert spaces
\n(RKHSs). We prove that when the RKHSs are universal, both functionals are zero if and only if the
\nrandom variables are pairwise independent. We also show that the kernel mutual information is an
\nupper bound near independence on the Parzen window estimate of the mutual information. Analogous
\nresults apply for two correlation-based dependence functionals introduced earlier: we show
\nthe kernel canonical correlation and the kernel generalised variance to be independence measures
\nfor universal kernels, and prove the latter to be an upper bound on the mutual information near
\nindependence. The performance of the kernel dependence functionals in measuring independence
\nis verified in the context of independent component analysis.