Event-Based, 6-DOF Camera Tracking from Photometric Depth Maps

TLDR

Event cameras output pixel‑level brightness changes, avoid motion blur, offer high dynamic range and low power, making them ideal for VR/AR and gaming. This work aims to achieve accurate, low‑latency tracking of an event camera using a pre‑existing photometric depth map. The method updates the 6‑DOF pose on each incoming event, virtually eliminating tracking latency. Experiments in indoor and outdoor scenes show the pipeline succeeds in high‑speed motion scenarios that standard cameras cannot handle.

Abstract

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. These features, along with a very low power consumption, make event cameras an ideal complement to standard cameras for VR/AR and video game applications. With these applications in mind, this paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map (i.e., intensity plus depth information) built via classic dense reconstruction pipelines. Our approach tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency. We successfully evaluate the method in both indoor and outdoor scenes and show that---because of the technological advantages of the event camera---our pipeline works in scenes characterized by high-speed motion, which are still unaccessible to standard cameras.