Regularized linear fuzzy clustering and probabilistic PCA mixture models

Abstract

Fuzzy c-means (FCM)-type fuzzy clustering approaches are closely related to Gaussian mixture models (GMMs) and EM-like algorithms have been used in FCM clustering with regularized objective functions. Especially, FCM with regularization by Kullback-Leibler information (KLFCM) is a fuzzy counterpart of GMMs. In this paper, we propose to apply probabilistic principal component analysis (PCA) mixture models to linear clustering following a discussion on the relationship between local PCA and linear fuzzy clustering. Although the proposed method is a kind of the constrained model of KLFCM, the algorithm includes the fuzzy c-varieties (FCV) algorithm as a special case, and the algorithm can be regarded as a modified FCV algorithm with regularization by K-L information. Numerical experiments demonstrate that the proposed clustering algorithm is more flexible than the maximum likelihood approaches and is useful for capturing local substructures properly.