Asynchronous Event-Based Binocular Stereo Matching

TLDR

Artificial retinas output continuous asynchronous temporal events, unlike conventional frame‑based cameras, mirroring biological retinal cells. The paper proposes a novel event‑based stereo matching algorithm that exploits asynchronous visual events from silicon retinas. The algorithm uses the timing of asynchronous events from silicon retinas to solve stereo matching for moving objects. Using high temporal resolution, the algorithm achieves real‑time 3‑D reconstruction of moving objects, accurately filtering incorrect matches and overcoming low spatial resolution, and demonstrates the importance of dynamic information and spike timing.

Abstract

We present a novel event-based stereo matching algorithm that exploits the asynchronous visual events from a pair of silicon retinas. Unlike conventional frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events, in a manner similar to the output cells of the biological retina. Our algorithm uses the timing information carried by this representation in addressing the stereo-matching problem on moving objects. Using the high temporal resolution of the acquired data stream for the dynamic vision sensor, we show that matching on the timing of the visual events provides a new solution to the real-time computation of 3-D objects when combined with geometric constraints using the distance to the epipolar lines. The proposed algorithm is able to filter out incorrect matches and to accurately reconstruct the depth of moving objects despite the low spatial resolution of the sensor. This brief sets up the principles for further event-based vision processing and demonstrates the importance of dynamic information and spike timing in processing asynchronous streams of visual events.