Semi-supervised weighted distance metric learning for kNN classification

Abstract

K-Nearest Neighbor (kNN) classification is one of the most popular machine learning techniques, but it often fails to work well due to less known information or inappropriate choice of distance metric or the presence of a lot of unrelated features. To handle those issues, we introduce a semi-supervised weighted distance metric learning method for kNN classification. This method uses a graph-based semi-supervised Label Propagation algorithm to gain more classification information with tiny initial classification information, then resorts to improved weighted Relevant Component Analysis to learn a Mahalanobis distance metric, and finally uses learned Mahalanobis distance metric to replace the original Euclidean distance of kNN classifier. Experiments on UCI datasets show the effectiveness of our method.