CAAI Transactions on Intelligence Technology · 2024 · 16 citations · 43 references
ComputingComputer ScienceComputer VisionFeature EngineeringSensor FusionAction RecognitionDeep LearningGraph Convolutional NetworksGraph-based Action RecognitionLate Fusion ArchitecturesMultimodal Recognition FusionPose-based Action RecognitionSemantic FeaturesSkeleton-based Action Recognition
Abstract Multimodal‐based action recognition methods have achieved high success using pose and RGB modality. However, skeletons sequences lack appearance depiction and RGB images suffer irrelevant noise due to modality limitations. To address this, the authors introduce human parsing feature map as a novel modality, since it can selectively retain effective semantic features of the body parts while filtering out most irrelevant noise. The authors propose a new dual‐branch framework called ensemble human parsing and pose network (EPP‐Net), which is the first to leverage both skeletons and human parsing modalities for action recognition. The first human pose branch feeds robust skeletons in the graph convolutional network to model pose features, while the second human parsing branch also leverages depictive parsing feature maps to model parsing features via convolutional backbones. The two high‐level features will be effectively combined through a late fusion strategy for better action recognition. Extensive experiments on NTU RGB + D and NTU RGB + D 120 benchmarks consistently verify the effectiveness of our proposed EPP‐Net, which outperforms the existing action recognition methods. Our code is available at https://github.com/liujf69/EPP‐Net‐Action .
43
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Rethinking the Inception Architecture for Computer Vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe et al. · 2016 · 30.9K citations
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi et al. · 2017 · 15.5K citations
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Amir Shahroudy, Jun Liu, Tian-Tsong Ng et al. · 2016 · 2.9K citations