Soft Sensor-Based Deep Temporal-Graph Convolutional Network for Applications in Human Motion Tracking

Abstract

The popularity of accurate real-time motion tracking wearables with multiple on-body soft stretchable sensors stems from their advantages of easy wearability, high stretchability, and low cost. Due to its high non-linearity and hysteresis, challenges remain in terms of the accuracy, calibration, and difficulty in estimating human motion from soft sensor signals. In order to address these issues, we present a novel deep learning-based calibration method for human motion tracking. Combining with temporal convolutional network (TCN), a weighted graph convolutional network (GCN) and the shortcut structure, we propose a Deep Temporal-Graph Convolutional Network with Pearson Correlation Coefficient (T-PGCN). The proposed T-PGCN is designed to extract the temporal features by a TCN module and the spatial features by a GCN module, subsequently, by means of a shortcut connection, the outputs and the original input data are fed into a fully connected neural network (FCNN) module to obtain the predicted positions at the tracking points. The effectiveness and generalization of our proposed T-PGCN model are demonstrated using two datasets: the DFM Motion Tracking dataset and the SSG Hand Posing Tracking dataset. The results show that T-PGCN performs better than the baseline models without increasing much training time.