Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars

Abstract

Event cameras are bio-inspired vision sensors that naturally
\ncapture the dynamics of a scene, filtering out redundant
\ninformation. This paper presents a deep neural
\nnetwork approach that unlocks the potential of event cameras
\non a challenging motion-estimation task: prediction
\nof a vehicle’s steering angle. To make the best out of this
\nsensor–algorithm combination, we adapt state-of-the-art
\nconvolutional architectures to the output of event sensors
\nand extensively evaluate the performance of our approach
\non a publicly available large scale event-camera dataset
\n(1000 km). We present qualitative and quantitative explanations
\nof why event cameras allow robust steering prediction
\neven in cases where traditional cameras fail, e.g. challenging
\nillumination conditions and fast motion. Finally, we
\ndemonstrate the advantages of leveraging transfer learning
\nfrom traditional to event-based vision, and show that our
\napproach outperforms state-of-the-art algorithms based on
\nstandard cameras.