A Survey of Multi-View Representation Learning

TLDR

Multi‑view representation learning is a rapidly growing area in machine learning and data mining. This survey categorizes multi‑view representation learning into alignment and fusion, aiming to provide an overview of theory and state‑of‑the‑art tools for researchers. The authors review alignment methods such as CCA and its extensions, and fusion approaches ranging from generative models to neural network‑based techniques, while also highlighting key applications.

Abstract

Recently, multi-view representation learning has become a rapidly growing direction in machine learning and data mining areas. This paper introduces two categories for multi-view representation learning: multi-view representation alignment and multi-view representation fusion. Consequently, we first review the representative methods and theories of multi-view representation learning based on the perspective of alignment, such as correlation-based alignment. Representative examples are canonical correlation analysis (CCA) and its several extensions. Then from the perspective of representation fusion we investigate the advancement of multi-view representation learning that ranges from generative methods including multi-modal topic learning, multi-view sparse coding, and multi-view latent space Markov networks, to neural network-based methods including multi-modal autoencoders, multi-view convolutional neural networks, and multi-modal recurrent neural networks. Further, we also investigate several important applications of multi-view representation learning. Overall, this survey aims to provide an insightful overview of theoretical foundation and state-of-the-art developments in the field of multi-view representation learning and to help researchers find the most appropriate tools for particular applications.