Proceedings of the 28th Annual International Conference on Mobile Computing And Networking · 2022 · 84 citations · 39 references
ComputingComputer ScienceComputer VisionMachine LearningSensor FusionAction RecognitionDeep LearningFeature-level FusionMultimodal InteractionAttentionAttention MechanismsAttention-based FusionContrastive LearningEdge CloudMultimodal Data FusionMultimodal Feature FusionMultimodal Fusion
Human activity recognition (HAR) is a key enabling technology for a wide range of emerging applications. Although multimodal sensing systems are essential for capturing complex and dynamic human activities in real-world settings, they bring several new challenges including limited labeled multimodal data. In this paper, we propose Cosmo, a new system for contrastive fusion learning with small data in multimodal HAR applications. Cosmo features a novel two-stage training strategy that leverages both unlabeled data on the cloud and limited labeled data on the edge. By integrating novel fusion-based contrastive learning and quality-guided attention mechanisms, Cosmo can effectively extract both consistent and complementary information across different modalities for efficient fusion. Our evaluation on a cloud-edge testbed using two public datasets and a new multimodal HAR dataset shows that Cosmo delivers significant improvement over state-of-the-art baselines in both recognition accuracy and convergence delay.
39
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
Unsupervised feature selection for multi-cluster data
Deng Cai, Chiyuan Zhang, Xiaofei He · 2010 · 1.1K citations