2021 · 18 citations · 22 references
ComputingComputer ScienceComputer VisionFood ProcessingInformation RetrievalImage RetrievalMultimedia RetrievalAdversarial LossCross-modal Representation LearningCross-modal RetrievalMulti-modal Representation LearningMulti-modal RetrievalMultimodal Representation LearningMultimodal RetrievalSubspace Learning
Cross-modal retrieval technology can help people quickly achieve mutual information between cooking recipes and food images. Both the embeddings of the image and the recipe consist of multiple representation subspaces. We argue that multiple aspects in the recipe are related to multiple regions in the food image. It is challenging to improve the cross-modal retrieval quality by making full use of the implicit connection between multiple subspaces of recipes and images. In this paper, we propose a multi-subspace implicit alignment cross-modal retrieval framework of recipes and images. Our framework learns multi-subspace information about cooking recipes and food images with multi-head attention networks; the implicit alignment at the subspace level promotes narrowing the semantic gap between recipe embeddings and food image embeddings; triple loss and adversarial loss are combined to help our framework for cross-modal learning. The experimental results show that our framework significantly outperforms to state-of-the-art methods in terms of MedR and [email protected] on Recipe 1M.
22
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
What Does BERT Learn about the Structure of Language?
Ganesh Jawahar, Benoît Sagot, Djamé Seddah · 2019 · 1.2K citations · Full text
Computing, Computer Science, English +17
Multi-view Subspace Clustering
Hongchang Gao, Feiping Nie, Xuelong Li et al. · 2015 · 619 citations · Full text
Learning Cross-Modal Embeddings for Cooking Recipes and Food Images
Amaia Salvador, Nicholas Hynes, Yusuf Aytar et al. · 2017 · 408 citations · Full text
Structured AutoEncoders for Subspace Clustering
Xi Peng, Jiashi Feng, Shijie Xiao et al. · IEEE Transactions on Image Processing · 2018 · 351 citations