A Multisensor Fusion With Automatic Vision–LiDAR Calibration Based on Factor Graph Joint Optimization for SLAM

Abstract

Combining multiple sensors for environment sensing and self-positioning is significant for automatic driving. This paper proposes a novel simultaneous localization and mapping (SLAM) system framework that integrates the information ofmultiple sensors including camera, Light Detection and Ranging (LiDAR), Inertial Measurement Unit (IMU), and Global Positioning System (GPS) based on vision-LiDAR calibration. Different sensors are fused in a tightly coupled manner and finally optimized by a factor graph. The automatic vision-LiDAR calibration (AVLC) is proposed in this paper to reduce the error caused by the unexpected change in the sensor. Further, the semantic map is established by the target detection module, which provides convenience for navigation and obstacle avoidance. The proposed algorithm uses the Complex-YOLO for 3D object recognition and then combines the recognition results with the semi-dense point cloud map generated by the multi-sensor fusion positioning algorithm with AVLC to achieve the purpose of enriching map information. Extensive experiments on multiple datasets show that the proposed algorithm has higher accuracy and robustness than other existing algorithms.