Improved covariance estimation for Gustafson-Kessel clustering

TLDR

GK clustering struggles with small sample sizes or linearly correlated data, and its use in extracting Takagi‑Sugeno fuzzy models also presents challenges. The paper proposes two techniques to improve the fuzzy covariance matrix calculation in the Gustafson‑Kessel clustering algorithm. The first technique fixes the eigenvalue ratio of the covariance matrix, while the second adds a scaled identity matrix to stabilize it. These methods lower overfitting risk when training samples are scarce relative to clusters and numerical experiments confirm their advantages.

Abstract

This article presents two techniques to improve the calculation of the fuzzy covariance matrix in the Gustafson-Kessel (GK) clustering algorithm. The first one overcomes problems that occur in the standard GK clustering when the number of data samples is small or when the data within a cluster are linearly correlated. The improvement is achieved by fixing the ratio between the maximal and minimal eigenvalue of the covariance matrix. The second technique is useful when the GK algorithm is employed in the extraction of Takagi-Sugeno fuzzy model from data. It reduces the risk of overfitting when the number of training samples is low in comparison to the number of clusters. This is achieved by adding a scaled unity matrix to the calculated covariance matrix. Numerical examples are presented to demonstrate the benefits of the proposed techniques.