ACM Transactions on Multimedia Computing Communications and Applications · 2021 · 20 citations · 31 references
ComputingComputer ScienceComputer VisionMachine LearningNatural Language ProcessingDeep LearningMultimedia RetrievalRelation ExtractionRepresentation LearningSocial MediaVisual Question AnsweringFeature FusionJoint Feature EmbeddingMultimodal EmbeddingMultimodal Feature FusionMultimodal FusionSemantic FeaturesVideo Content AnalysisVideo Retrieval
As a crucial task for video analysis, social relation recognition for characters not only provides semantically rich description of video content but also supports intelligent applications, e.g., video retrieval and visual question answering. Unfortunately, due to the semantic gap between visual and semantic features, traditional solutions may fail to reveal the accurate relations among characters. At the same time, the development of social media platforms has now promoted the emergence of crowdsourced comments, which may enhance the recognition task with semantic and descriptive cues. To that end, in this article, we propose a novel multimodal-based solution to deal with the character relation recognition task. Specifically, we capture the target character pairs via a search module and then design a multistream architecture for jointly embedding the visual and textual information, in which feature fusion and attention mechanism are adapted for better integrating the multimodal inputs. Finally, supervised learning is applied to classify character relations. Experiments on real-world data sets validate that our solution outperforms several competitive baselines.
31
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Person re-identification by Local Maximal Occurrence representation and metric learning
Shengcai Liao, Yang Hu, Xiangyu Zhu et al. · 2015 · 2.2K citations
CityPersons: A Diverse Dataset for Pedestrian Detection
Shanshan Zhang, Rodrigo Benenson, Bernt Schiele · 2017 · 989 citations