v2e: From Video Frames to Realistic DVS Events

TLDR

Realistic DVS events are useful for training networks under uncontrolled lighting conditions. The authors propose v2e, a toolbox that generates realistic synthetic DVS events from intensity frames and corrects misconceptions about DVS motion blur and latency. v2e incorporates pixel‑level Gaussian threshold mismatch, intensity‑dependent bandwidth, and noise, and is evaluated in two experiments: object recognition on N‑Caltech 101 and night‑time car detection. Pretraining on v2e lighting conditions improves ResNet generalization on real DVS data, and a v2e‑trained car detector outperforms YOLOv3 by 40 % on night driving.

Abstract

To help meet the increasing need for dynamic vision sensor (DVS) event camera data, this paper proposes the v2e toolbox that generates realistic synthetic DVS events from intensity frames. It also clarifies incorrect claims about DVS motion blur and latency characteristics in recent literature. Unlike other toolboxes, v2e includes pixel-level Gaussian event threshold mismatch, finite intensity-dependent bandwidth, and intensity-dependent noise. Realistic DVS events are useful in training networks for uncontrolled lighting conditions. The use of v2e synthetic events is demonstrated in two experiments. The first experiment is object recognition with N-Caltech 101 dataset. Results show that pretraining on various v2e lighting conditions improves generalization when transferred on real DVS data for a ResNet model. The second experiment shows that for night driving, a car detector trained with v2e events shows an average accuracy improvement of 40% compared to the YOLOv3 trained on intensity frames.