LIDAR and vision-based real-time traffic sign detection and recognition algorithm for intelligent vehicle

Abstract

Real-time traffic sign detection and recognition are essential and challenging tasks for intelligent vehicle. The previous works mainly focus on detecting and recognizing traffic sign based on images captured by onboard camera. Visual features of traffic sign such as color, shape, and appearance, however, are often sensitive to illumination condition, angle of view, etc. Except for camera, LIDAR also provides important and alternative features of traffic sign. The fusion of complementary data acquired from both sensors can improve the robustness of the algorithm, especially when data from either of them are of low quality. For this reason, we propose a new traffic sign detection and recognition algorithm based on the fusion of camera and LIDAR data. Specifically, position prior, color, laser reflectivity, and 3D geometric features are integrated to detect traffic sign in a 3D space. In most of previous works, different colors of traffic signs are individually handled in a specific color space, which generally results in the use of many thresholds or multiple classifiers. In this paper, we use combined color spaces (CCS) such that traffic sign colors can be entirely treated as one class. For traffic sign recognition, in order to improve the robustness to any viewpoint variation, regions of interest (ROIs), which suffer from perspective deformation, are rectified first by fusing LIDAR and camera data. Then the histogram of oriented gradient (HOG) feature and linear support vector machines (SVMs) are used to classify traffic signs. Finally, extensive experimental results in challenging conditions show that our algorithm is real-time and robust.