Closed-form cauchy-schwarz PDF divergence for mixture of Gaussians

Abstract

This paper presents an efficient approach to calculate the difference between two probability density functions (pdfs), each of which is a mixture of Gaussians (MoG). Unlike Kullback-Leibler divergence (D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">KL</sub> ), the authors propose that the Cauchy-Schwarz (CS) pdf divergence measure (D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CS</sub> ) can give an analytic, closed-form expression for MoG. This property of the D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CS</sub> makes fast and efficient calculations possible, which is tremendously desired in real-world applications where the dimensionality of the data/features is very high. We show that D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CS</sub> follows similar trends to D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">KL</sub> , but can be computed much faster, especially when the dimensionality is high. Moreover, the proposed method is shown to significantly outperform D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">KL</sub> in classifying real-world 2D and 3D objects, and static hand posture recognition based on distances alone.