LiDAR and Camera Detection Fusion in a Real-Time Industrial Multi-Sensor Collision Avoidance System

Pan Wei, Lucas Cagle, Tasmia Reza, John E. Ball, James Gafford

Electronics · 2018 · 124 citations · 32 references

DOIFull text

Open access

Concepts

TL;DR

Collision avoidance is essential in ADAS, industrial automation, and robotics, where certain zones must be kept off‑limits and can be delineated by mapping or passive beacons. The study proposes a method that uses LiDAR and a single color camera to detect passive beacons and employ model‑predictive control to prevent an automated vehicle from entering restricted areas. The system fuses LiDAR detections of orange traffic‑cone beacons with deep‑learning‑based camera projections and a neural‑network‑derived mapping to validate detections, thereby reducing false positives. Experiments at Mississippi State University’s CAVS demonstrate that the fusion approach maintains true detections while significantly reducing false positives.

Abstract

Collision avoidance is a critical task in many applications, such as ADAS (advanced driver-assistance systems), industrial automation and robotics. In an industrial automation setting, certain areas should be off limits to an automated vehicle for protection of people and high-valued assets. These areas can be quarantined by mapping (e.g., GPS) or via beacons that delineate a no-entry area. We propose a delineation method where the industrial vehicle utilizes a LiDAR (Light Detection and Ranging) and a single color camera to detect passive beacons and model-predictive control to stop the vehicle from entering a restricted space. The beacons are standard orange traffic cones with a highly reflective vertical pole attached. The LiDAR can readily detect these beacons, but suffers from false positives due to other reflective surfaces such as worker safety vests. Herein, we put forth a method for reducing false positive detection from the LiDAR by projecting the beacons in the camera imagery via a deep learning method and validating the detection using a neural network-learned projection from the camera to the LiDAR space. Experimental data collected at Mississippi State University’s Center for Advanced Vehicular Systems (CAVS) shows the effectiveness of the proposed system in keeping the true detection while mitigating false positives.

References

32