2014 · 130 citations · 18 references
Event-based VisionEvent CameraEngineeringField RoboticsLocalizationProcessing PipelineRobot LearningKinematicsVision SensorCartographyMachine VisionVision RoboticsVehicle LocalizationStructure From MotionNormal Cmos CameraComputer VisionOdometryEye TrackingPrevious Cmos FrameRobotics
Robotic agility is limited by processing speed, yet Dynamic Vision Sensors can deliver microsecond latency by outputting asynchronous luminance‑change events, though they lack grayscale values and produce event streams that render traditional frame‑based visual odometry inapplicable. This work introduces the first visual odometry system that fuses a DVS with a conventional CMOS camera to recover absolute brightness. The system automatically spatiotemporally calibrates the two sensors from operational logs and estimates relative displacement between consecutive CMOS frames by processing each DVS event individually. Experiments demonstrate that rotation can be estimated with surprising accuracy, while translation is noisy due to the scarcity of events from small apparent motion.
The agility of a robotic system is ultimately limited by the speed of its processing pipeline. The use of a Dynamic Vision Sensors (DVS), a sensor producing asynchronous events as luminance changes are perceived by its pixels, makes it possible to have a sensing pipeline of a theoretical latency of a few microseconds. However, several challenges must be overcome: a DVS does not provide the grayscale value but only changes in the luminance; and because the output is composed by a sequence of events, traditional frame-based visual odometry methods are not applicable. This paper presents the first visual odometry system based on a DVS plus a normal CMOS camera to provide the absolute brightness values. The two sources of data are automatically spatiotemporally calibrated from logs taken during normal operation. We design a visual odometry method that uses the DVS events to estimate the relative displacement since the previous CMOS frame by processing each event individually. Experiments show that the rotation can be estimated with surprising accuracy, while the translation can be estimated only very noisily, because it produces few events due to very small apparent motion.
18
Sebastian Thrun · Communications of the ACM · 2002 · 7.9K citations
Artificial Intelligence, Path Planning, Imperfect Real-world Environments +13
Davide Scaramuzza, Friedrich Fraundorfer · IEEE Robotics & Automation Magazine · 2011 · 1.5K citations