<i>R</i>₁-PCA

TLDR

Principal component analysis (PCA) minimizes the sum of squared errors (L2‑norm) and is sensitive to outliers. The authors propose a rotationally invariant L1‑norm PCA (R1‑PCA). R1‑PCA has a unique global solution, uses principal eigenvectors of a re‑weighted robust covariance matrix, is rotationally invariant, and is computed efficiently with a new subspace iteration algorithm. R1‑PCA’s properties distinguish it from standard L1‑PCA, experiments on real datasets show it effectively handles outliers, and extending the approach to K‑means yields R1‑K‑means that outperforms standard K‑means.

Abstract

Principal component analysis (PCA) minimizes the sum of squared errors (L2-norm) and is sensitive to the presence of outliers. We propose a rotational invariant L1-norm PCA (R1-PCA). R1-PCA is similar to PCA in that (1) it has a unique global solution, (2) the solution are principal eigenvectors of a robust covariance matrix (re-weighted to soften the effects of outliers), (3) the solution is rotational invariant. These properties are not shared by the L1-norm PCA. A new subspace iteration algorithm is given to compute R1-PCA efficiently. Experiments on several real-life datasets show R1-PCA can effectively handle outliers. We extend R1-norm to K-means clustering and show that L1-norm K-means leads to poor results while R1-K-means outperforms standard K-means.