Event-Based Visual Inertial Odometry

TLDR

Event‑based cameras asynchronously detect intensity changes across all pixels, delivering up to 1 MHz event rates that enable high‑speed, high‑dynamic‑range sensing beyond the limits of conventional cameras. This work introduces the first algorithm that fuses a purely event‑based tracker with an inertial measurement unit to achieve accurate metric 6‑DOF pose estimation. The asynchronous method selects image‑plane features, tracks spatiotemporal windows in the event stream, and fuses these tracks with IMU outputs via an Extended Kalman Filter with a structureless measurement model, using the resulting poses to initialize subsequent tracking steps and reject failures. On the Event‑Camera Dataset, the proposed approach reliably tracks camera motion under a variety of challenging conditions.

Abstract

Event-based cameras provide a new visual sensing model by detecting changes in image intensity asynchronously across all pixels on the camera. By providing these events at extremely high rates (up to 1MHz), they allow for sensing in both high speed and high dynamic range situations where traditional cameras may fail. In this paper, we present the first algorithm to fuse a purely event-based tracking algorithm with an inertial measurement unit, to provide accurate metric tracking of a cameras full 6dof pose. Our algorithm is asynchronous, and provides measurement updates at a rate proportional to the camera velocity. The algorithm selects features in the image plane, and tracks spatiotemporal windows around these features within the event stream. An Extended Kalman Filter with a structureless measurement model then fuses the feature tracks with the output of the IMU. The camera poses from the filter are then used to initialize the next step of the tracker and reject failed tracks. We show that our method successfully tracks camera motion on the Event-Camera Dataset in a number of challenging situations.