SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud

Abstract

4D LiDAR semantic segmentation classifies the semantic category of each LiDAR point and detects whether it is dynamic, a critical ability for tasks like obstacle avoidance and autonomous navigation. Existing approaches often rely on computationally heavy 4D convolutions or recursive networks, which result in poor real-time performance. In this paper, we introduce SegNet4D, a novel real-time 4D semantic segmentation network, offering both efficiency and strong semantic understanding. SegNet4D addresses 4D segmentation as two tasks: single-scan semantic segmentation and moving object segmentation, each tackled by a separate network head. Both results are combined in a motion-semantic fusion module to achieve comprehensive 4D segmentation. Additionally, instance information is extracted from the current scan and exploited for instance-wise segmentation consistency. Extensive experiments on the SemanticKITTI and nuScenes datasets demonstrate that our method outperforms the state-of-the-art in both 4D semantic segmentation and moving object segmentation. Through detailed runtime analysis, our method shows greater efficiency, enabling real-time operation. Besides, its effectiveness and efficiency have also been validated on a real-world robotic platform. The implementation of our method has been released at https: //github.com/nubot-nudt/SegNet4D.