Toward Visual Interaction: Hand Segmentation by Combining 3-D Graph Deep Learning and Laser Point Cloud for Intelligent Rehabilitation

Abstract

Against the backdrop of the increasing trend of aging population in China and even globally, the demand for hand function rehabilitation is growing day by day, and human-machine interaction virtual rehabilitation systems have become a research hotspot. Currently, 3-D vision has shown great potential in morphological analysis, but the complexity and irregularity of hand surfaces pose challenges for accurate segmentation. This study has proposed a hand surface segmentation network (HSSN) for intelligent hand function rehabilitation in the virtual reality, which combines 3-D graph deep learning and laser point cloud. HSSN integrates a series of methods, with edge convolution layers effectively addressing the complex morphology of hand surfaces, multiscale edge convolution solving the problem of missing or redundant local features, multidensity processing enhancing the robustness of the model to point cloud density, and normal vector feature enhancement solving the problem of insufficient geometric features of actual hand surface point clouds. Through the comprehensive application of these methods, HSSN has demonstrated excellent performance in comparative experiments. This study is of great significance for promoting the personalized and precise development of intelligent rehabilitation in the virtual reality environment. More importantly, this achievement has provided a new perspective for interdisciplinary research in fields, such as rehabilitation engineering and human-machine interaction.