Fast Image Reconstruction with an Event Camera

TLDR

Event cameras capture high dynamic range with microsecond resolution and no motion blur by detecting brightness changes, and prior image reconstruction methods rely on hand‑crafted spatial and temporal smoothing. The authors aim to develop a compact neural network for event‑based video reconstruction that is smaller and faster than existing methods while preserving performance. They design a neural architecture with only 38 k parameters, enabling a 10 ms.

Abstract

Event cameras are powerful new sensors able to capture high dynamic range with microsecond temporal resolution and no motion blur. Their strength is detecting brightness changes (called events) rather than capturing direct brightness images; however, algorithms can be used to convert events into usable image representations for applications such as classification. Previous works rely on hand-crafted spatial and temporal smoothing techniques to reconstruct images from events. State-of-the-art video reconstruction has recently been achieved using neural networks that are large (10M parameters) and computationally expensive, requiring 30ms for a forward-pass at 640 × 480 resolution on a modern GPU. We propose a novel neural network architecture for video reconstruction from events that is smaller (38k vs. 10M parameters) and faster (10ms vs. 30ms) than state-of-the-art with minimal impact to performance.