arXiv (Cornell University) · 2020 · 20 citations · 16 references
ComputingComputer ScienceMachine LearningProbability TheoryStatistical LearningDeep LearningGenerative ModelingNeural NetworksRepresentation LearningGaussian Mixture ModelsGenerative Adversarial NetworksMixture ModelMomentsRandom Matrix TheoryStatistical Moments
This paper shows that deep learning (DL) representations of data produced by generative adversarial nets (GANs) are random vectors which fall within the class of so-called \textit{concentrated} random vectors. Further exploiting the fact that Gram matrices, of the type $G = X^T X$ with $X=[x_1,\ldots,x_n]\in \mathbb{R}^{p\times n}$ and $x_i$ independent concentrated random vectors from a mixture model, behave asymptotically (as $n,p\to \infty$) as if the $x_i$ were drawn from a Gaussian mixture, suggests that DL representations of GAN-data can be fully described by their first two statistical moments for a wide range of standard classifiers. Our theoretical findings are validated by generating images with the BigGAN model and across different popular deep representation networks.
16
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Learning representations by back-propagating errors
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams · Nature · 1986 · 30.9K citations
Conditional Generative Adversarial Nets
Mehdi Mirza, Simon Osindero · arXiv (Cornell University) · 2014 · 8.9K citations · Full text
On Spectral Clustering: Analysis and an algorithm
Andrew Y. Ng, Michael I. Jordan, Yair Weiss · 2001 · 7.8K citations