Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception

TLDR

Event‑based neuromorphic vision sensors, inspired by the retina, offer low‑energy, low‑latency, high‑dynamic‑range, high‑temporal‑resolution sensing by asynchronously capturing pixel‑level light changes, marking a paradigm shift from frame‑based cameras to advanced autonomous‑vehicle visual systems. This tutorial‑style review surveys the emerging neuromorphic vision technology, outlines its challenges and future research directions, and aims to provide a starting point for researchers and engineers in autonomous driving. The article traces the sensor’s evolution from retinal principles, discusses noise‑reduction and data‑representation techniques, and reviews signal‑processing algorithms and applications for autonomous driving and assistance systems.

Abstract

As a bio-inspired and emerging sensor, an event-based neuromorphic vision sensor has a different working principle compared to the standard frame-based cameras, which leads to promising properties of low energy consumption, low latency, high dynamic range (HDR), and high temporal resolution. It poses a paradigm shift to sense and perceive the environment by capturing local pixel-level light intensity changes and producing asynchronous event streams. Advanced technologies for the visual sensing system of autonomous vehicles from standard computer vision to event-based neuromorphic vision have been developed. In this tutorial-like article, a comprehensive review of the emerging technology is given. First, the course of the development of the neuromorphic vision sensor that is derived from the understanding of biological retina is introduced. The signal processing techniques for event noise processing and event data representation are then discussed. Next, the signal processing algorithms and applications for event-based neuromorphic vision in autonomous driving and various assistance systems are reviewed. Finally, challenges and future research directions are pointed out. It is expected that this article will serve as a starting point for new researchers and engineers in the autonomous driving field and provide a bird's-eye view to both neuromorphic vision and autonomous driving research communities.