2023 · 28 citations · 42 references
ComputingComputer ScienceComputer VisionNatural Language ProcessingVisual AttentionAction RecognitionAction RepresentationScene UnderstandingVideo AnalysisAction DetectionAttentionSalient Feature IdentificationSalient Region DetectionSemantic ConsistencyTemporal Action LocalizationVideo Action RecognitionWeakly Supervised Action Recognition
Weakly-supervised temporal action localization aims to detect action boundaries in untrimmed videos with only video-level annotations. Most existing schemes detect tem-poral regions that are most responsive to video-level classification, but they overlook the semantic consistency between frames. In this paper, we hypothesize that snippets with similar representations should be considered as the same action class despite the absence of supervision signals on each snippet. To this end, we devise a learnable dictionary where entries are the class centroids of the corresponding action categories. The representations of snippets identified as the same action category are induced to be close to the same class centroid, which guides the network to perceive the semantics of frames and avoid unreasonable localization. Besides, we propose a two-stream framework that in-tegrates the attention mechanism and the multiple-instance learning strategy to extract fine-grained clues and salient features respectively. Their complementarity enables the model to refine temporal boundaries. Finally, the developed model is validated on the publicly available THUMOS-14 and ActivityNet-1.3 datasets, where substantial experiments and analyses demonstrate that our model achieves remark-able advances over existing methods.
42
SlowFast Networks for Video Recognition
Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik et al. · 2019 · 3.6K citations
The Kinetics Human Action Video Dataset
Will Kay, João Carreira, Karen Simonyan et al. · arXiv (Cornell University) · 2017 · 2.9K citations · Full text
ActivityNet: A large-scale video benchmark for human activity understanding
Fabian Caba Heilbron, Víctor Escorcia, Bernard Ghanem et al. · 2015 · 2.6K citations