ACM Transactions on Information Systems · 2023 · 17 citations · 48 references
ComputingComputer ScienceComputer VisionKnowledge RepresentationNatural Language ProcessingDeep LearningModal SemanticsMultimodal InteractionScene UnderstandingSemantic NetworksCross-modal FusionCross-modal LocalizationMultimodal FusionMultimodal Representation LearningNatural Language QueryingNetwork RepresentationSemantic AlignmentSemantic Information
Localizing a desired moment within an untrimmed video via a given natural language query, i.e., cross-modal moment localization, has attracted widespread research attention recently. However, it is a challenging task because it requires not only accurately understanding intra-modal semantic information, but also explicitly capturing inter-modal semantic correlations (consistency and complementarity). Existing efforts mainly focus on intra-modal semantic understanding and inter-modal semantic alignment, while ignoring necessary semantic supplement. Consequently, we present a cross-modal semantic perception network for more effective intra-modal semantic understanding and inter-modal semantic collaboration. Concretely, we design a dual-path representation network for intra-modal semantic modeling. Meanwhile, we develop a semantic collaborative network to achieve multi-granularity semantic alignment and hierarchical semantic supplement. Thereby, effective moment localization can be achieved based on sufficient semantic collaborative learning. Extensive comparison experiments demonstrate the promising performance of our model compared with existing state-of-the-art competitors.
48
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Shaoqing Ren, Kaiming He, Ross Girshick et al. · arXiv (Cornell University) · 2015 · 18.2K citations · Full text
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
1993 · 14.3K citations
Proceedings of IEEE International Conference on Computer Vision
1995 · 12.9K citations
Learning Spatiotemporal Features with 3D Convolutional Networks
Du Tran, Lubomir Bourdev, Rob Fergus et al. · 2015 · 9.7K citations