Principal-component-analysis eigenvalue spectra from data with symmetry-breaking structure

Abstract

Principal component analysis (PCA) is a ubiquitous method of multivariate statistics that focuses on the eigenvalues $\ensuremath{\lambda}$ and eigenvectors of the sample covariance matrix of a data set. We consider p, N-dimensional data vectors $\mathit{\ensuremath{\xi}}$ drawn from a distribution with covariance matrix $\mathbf{C}.$ We use the replica method to evaluate the expected eigenvalue distribution $\ensuremath{\rho}(\ensuremath{\lambda})$ as $\stackrel{\ensuremath{\rightarrow}}{N}\ensuremath{\infty}$ with $p=\ensuremath{\alpha}N$ for some fixed $\ensuremath{\alpha}.$ In contrast to existing studies we consider the case where $\mathbf{C}$ contains a number of symmetry-breaking directions, so that the sample data set contains some definite structure. Explicitly we set $\mathbf{C}={\ensuremath{\sigma}}^{2}\mathbf{I}+{\ensuremath{\sigma}}^{2}{\ensuremath{\sum}}_{m=1}^{S}{A}_{m}{\mathbf{B}}_{m}{\mathbf{B}}_{m}^{T},$ with ${A}_{m}>0\ensuremath{\forall}m.$ We find that the bulk of the eigenvalues are distributed as for the case when the elements of $\mathit{\ensuremath{\xi}}$ are independent and identically distributed. With increasing $\ensuremath{\alpha}$ a series of phase transitions are observed, at $\ensuremath{\alpha}{=A}_{m}^{\ensuremath{-}2},m=1,2,\dots{},S,$ each time a single $\ensuremath{\delta}$ function, $\ensuremath{\delta}(\ensuremath{\lambda}\ensuremath{-}{\ensuremath{\lambda}}_{u}{(A}_{m})),$ separates from the upper edge of the bulk distribution, where ${\ensuremath{\lambda}}_{u}(A)={\ensuremath{\sigma}}^{2}[1+A][1+(\ensuremath{\alpha}{A)}^{\ensuremath{-}1}].$ We confirm the results of the replica analysis by studying the Stieltjes transform of $\ensuremath{\rho}(\ensuremath{\lambda}).$ This suggests that the results obtained from the replica analysis are universal, irrespective of the distribution from which $\mathit{\ensuremath{\xi}}$ is drawn, provided the fourth moment of each element of $\mathit{\ensuremath{\xi}}$ exists.