IEEE Sensors Journal · 2021 · 51 citations · 49 references
ComputingComputer ScienceComputer VisionMachine LearningSensor FusionAction RecognitionDeep LearningFeature-level FusionImage FusionActivity RecognitionCanonical Correlation AnalysisDeep Image FeaturesHuman ActionImage Feature FusionMultimodal Feature FusionMultimodal FusionMultimodal Sensor Fusion
Convolutional Neural Networks (CNNs) are successful deep learning models in the field of computer vision. To get the maximum advantage of CNN model for Human Action Recognition (HAR) using inertial sensor data, in this paper, we use four types of spatial domain methods for transforming inertial sensor data to activity images, which are then utilized in a novel fusion framework. These four types of activity images are Signal Images (SI), Gramian Angular Field (GAF) Images, Markov Transition Field (MTF) Images and Recurrence Plot (RP) Images. Furthermore, for creating a multimodal fusion framework and to exploit activity images, we made each type of activity images multimodal by convolving with two spatial domain filters: Prewitt filter and High-boost filter. ResNet-18, a CNN model, is used to learn deep features from multi-modalities. Learned features are extracted from the last pooling layer of each ResNet and then fused by canonical correlation based fusion (CCF) for improving the accuracy of human action recognition. These highly informative features are served as input to a multi-class Support Vector Machine (SVM). Experimental results on three publicly available inertial datasets show the superiority of the proposed method over the current state-of-the-art.
49
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
Deep learning for sensor-based activity recognition: A survey
Jindong Wang, Yiqiang Chen, Shuji Hao et al. · Pattern Recognition Letters · 2018 · 1.9K citations · Full text