Event-based 3D SLAM with a depth-augmented dynamic vision sensor

TLDR

The D‑eDVS combines a PrimeSense RGB‑D sensor with a biologically inspired dynamic vision sensor that reacts to contrast changes and outputs a sparse stream of events representing pixel locations. The paper introduces the D‑eDVS, a combined event‑based 3D sensor, and a novel full‑3D SLAM algorithm that operates solely on its sparse event stream. The method fuses the event‑based sensor with a classic RGB‑D camera to generate a sparse, depth‑augmented 3‑D point stream, yielding lower data volume, efficient resource use, and continuous, lag‑free motion tracking. The algorithm achieves real‑time performance, running 20× faster than real time, delivering several hundred hertz localization updates, and outperforms two state‑of‑the‑art methods on a newly released dataset.

Abstract

We present the D-eDVS- a combined event-based 3D sensor - and a novel event-based full-3D simultaneous localization and mapping algorithm which works exclusively with the sparse stream of visual data provided by the D-eDVS. The D-eDVS is a combination of the established PrimeSense RGB-D sensor and a biologically inspired embedded dynamic vision sensor. Dynamic vision sensors only react to dynamic contrast changes and output data in form of a sparse stream of events which represent individual pixel locations. We demonstrate how an event-based dynamic vision sensor can be fused with a classic frame-based RGB-D sensor to produce a sparse stream of depth-augmented 3D points. The advantages of a sparse, event-based stream are a much smaller amount of generated data, thus more efficient resource usage, and a continuous representation of motion allowing lag-free tracking. Our event-based SLAM algorithm is highly efficient and runs 20 times faster than realtime, provides localization updates at several hundred Hertz, and produces excellent results. We compare our method against ground truth from an external tracking system and two state-of-the-art algorithms on a new dataset which we release in combination with this paper.