Proceedings of the AAAI Conference on Artificial Intelligence · 2020 · 10 citations · 27 references
ComputingComputer ScienceComputer VisionMachine LearningSensor FusionDeep LearningLatent Variable ModelsMultisensory IntegrationNeural NetworksUnsupervised LearningVariational InferenceAutoencodersCross-modal FusionCross-modal Representation LearningHeterogeneous Data FusionLatent Space EmbeddingMulti-modal Representation LearningMultimodal Machine LearningMultimodal Representation Learning
In this paper, we propose a novel structure for a multi-modal data association referred to as Associative Variational Auto-Encoder (AVAE). In contrast to the existing models using a shared latent space among modalities, our structure adopts distributed latent spaces for multi-modalities which are connected through cross-modal associators. The proposed structure successfully associates even heterogeneous modality data and easily incorporates the additional modality to the entire network via the associator. Furthermore, in our structure, only a small amount of supervised (paired) data is enough to train associators after training auto-encoders in an unsupervised manner. Through experiments, the effectiveness of the proposed structure is validated on various datasets including visual and auditory data.
27
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Auto-Encoding Variational Bayes
Diederik P. Kingma, Max Welling · UvA-DARE (University of Amsterdam) · 2013 · 15.6K citations · Full text
Reading digits in natural images with unsupervised feature learning
Yuval Netzer · 2024 · 4.6K citations · Full text