arXiv (Cornell University) · 2017 · 36 citations · 20 references
ComputingComputer ScienceComputer VisionFashion DesignInformation RetrievalFashionImage RetrievalSemantic NetworksVisual SemanticsEmbedding Learning For RetrievalImage-text RetrievalLatent Space EmbeddingSemantic Knowledge RepresentationSpatial Semantic RepresentationVisual Concept EmbeddingVisual Concept LearningVisual-spatial Representation
This paper proposes an automatic spatially-aware concept discovery approach using weakly labeled image-text data from shopping websites. We first fine-tune GoogleNet by jointly modeling clothing images and their corresponding descriptions in a visual-semantic embedding space. Then, for each attribute (word), we generate its spatially-aware representation by combining its semantic word vector representation with its spatial representation derived from the convolutional maps of the fine-tuned network. The resulting spatially-aware representations are further used to cluster attributes into multiple groups to form spatially-aware concepts (e.g., the neckline concept might consist of attributes like v-neck, round-neck, etc). Finally, we decompose the visual-semantic embedding space into multiple concept-specific subspaces, which facilitates structured browsing and attribute-feedback product retrieval by exploiting multimodal linguistic regularities. We conducted extensive experiments on our newly collected Fashion200K dataset, and results on clustering quality evaluation and attribute-feedback product retrieval task demonstrate the effectiveness of our automatically discovered spatially-aware concepts.
20
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
Efficient Estimation of Word Representations in Vector Space
Tomáš Mikolov, Kai Chen, Greg S. Corrado et al. · arXiv (Cornell University) · 2013 · 11.7K citations · Full text
Learning Deep Features for Discriminative Localization
Bolei Zhou, Aditya Khosla, Àgata Lapedriza et al. · 2016 · 10.8K citations · Full text
DeViSE: A Deep Visual-Semantic Embedding Model
Andrea Frome, Greg S. Corrado, Jon Shlens et al. · 2013 · 2.1K citations
DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations