IEEE Transactions on Circuits and Systems for Video Technology · 2023 · 11 citations · 42 references
ComputingComputer ScienceComputational LinguisticsCrimeNatural Language ProcessingAction RecognitionAction RepresentationAnnotationEvent DetectionEvent RepresentationSequence LabelingCo-occurrence ModelingEvent UnderstandingMulti-label AnnotationPlug-and-play ModulesSemantic InformationSequence Representation LearningTemporal Action LocalizationTemporal Action Representation
Temporal action localization (TAL) is a prevailing task due to its great application potential. Existing works in this field mainly suffer from two weaknesses: (1) They often neglect the multi-label case and only focus on temporal modeling. (2) They ignore the semantic information in class labels and only use the visual information. To solve these problems, we propose a novel Co-Occurrence Relation Module (CORM) that explicitly models the co-occurrence relationship between actions. Besides the visual information, it further utilizes the semantic embeddings of class labels to model the co-occurrence relationship. The CORM works in a plug-and-play manner and can be easily incorporated with the existing sequence models. By considering both visual and semantic co-occurrence, our method achieves high multi-label relationship modeling capacity. Meanwhile, existing datasets in TAL always focus on low-semantic atomic actions. Thus we construct a challenging multi-label dataset UCF-Crime-TAL that focuses on high-semantic actions by annotating the UCF-Crime dataset at frame level and considering the semantic overlap of different events. Extensive experiments on two commonly used TAL datasets, i.e., MultiTHUMOS and TSU, and our newly proposed UCF-Crime-TAL demenstrate the effectiveness of the proposed CORM, which achieves state-of-the-art performance on these datasets.
42
ActivityNet: A large-scale video benchmark for human activity understanding
Fabian Caba Heilbron, Víctor Escorcia, Bernard Ghanem et al. · 2015 · 2.6K citations
Temporal Convolutional Networks for Action Segmentation and Detection
Colin Lea, M. D. Flynn, René Vidal et al. · 2017 · 2K citations