2005 · 954 citations · 29 references
ComputingData ScienceData MiningMachine LearningStatistical ModelingDimension ReductionDimensionality ReductionGaussian Process ModelingGaussian ProcessesLatent Variable ModelingLatent Variable ModelsGaussian Process Latent Variable ModelsLatent Gaussian ModelsLow-dimensional EmbeddingsMultivariate Gaussian Processes
Summarising a high dimensional data set with a low dimensional embedding is a standard approach for exploring its structure. In this paper we provide an overview of some existing techniques for discovering such embeddings. We then introduce a novel probabilistic interpretation of principal component analysis (PCA) that we term dual probabilistic PCA (DPPCA). The DPPCA model has the additional advantage that the linear mappings from the embedded space can easily be nonlinearised through Gaussian processes. We refer to this model as a Gaussian process latent variable model (GP-LVM). Through analysis of the GP-LVM objective function, we relate the model to popular spectral techniques such as kernel PCA and multidimensional scaling. We then review a practical algorithm for GP-LVMs in the context of large data sets and develop it to also handle discrete valued data and missing attributes. We demonstrate the model on a range of real-world and artificially generated data sets.
29
On Information and Sufficiency
S. Kullback, R. A. Leibler · The Annals of Mathematical Statistics · 1951 · 20K citations · Full text
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Sam T. Roweis, Lawrence K. Saul · Science · 2000 · 15K citations
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Joshua B. Tenenbaum, Vin de Silva, John Langford · Science · 2000 · 13.7K citations
Teuvo Kohonen · Proceedings of the IEEE · 1990 · 8.2K citations