Capturing Human Pose Using mmWave Radar

Abstract

Human pose estimation is an important task. Traditional human pose capturing systems are based on images or videos, which may suffer from bad light and raise the concerns of privacy. In this paper, we proposed an accurate human pose estimation system using the 77GHz millimeter wave radar. It is the first time that people use off-the-shelf millimeter wave radar to complete such a task. Our system requires no camera or specific sensors on the body to estimate the human skeleton. The system first uses two radar data to generate heatmaps and then adopts CNN to transform two-dimensional heatmaps into human pose. We use coordinated heatmaps from radar and visual inputs extracting from camera together to train the designed network. Based on our dataset and system, the proposed method achieves an average OKS value of 0.705 and 0.877 in AP 50.