Low-latency visual odometry using event-based feature tracks

TLDR

Vision sensors such as DAVIS combine a conventional camera and an event‑based sensor, offering low latency, high temporal resolution, and high dynamic range, but require new algorithms to handle their asynchronous event streams and synchronous frames. This study introduces a low‑latency visual odometry algorithm for the DAVIS sensor that uses event‑based feature tracks. The algorithm first detects features in grayscale frames, tracks them asynchronously with the event stream, and then feeds the tracks into an event‑based visual odometry pipeline that tightly interleaves robust pose optimization with probabilistic mapping. The approach accurately estimates the sensor’s 6‑DOF motion in natural scenes and represents the first event‑based visual odometry work on DAVIS using feature tracks.

Abstract

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. In this paper, we present a low-latency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks.