Impression Network for Video Object Detection

Abstract

Object detection in videos is more challenging than in static images. It’s proved that naively applying object detector frame by frame can be slow and inaccurate - visual clues can get weakened by defocus and motion blur, causing failure on corresponding frames. Multi-frame feature fusion methods proved effective in improving the accuracy, but they largely sacrifice the speed. Feature propagation methods proved effective in improving the speed, at the cost of lower accuracy. So is it possible to improve speed and performance at once?Inspired by how human utilize impression to recognize objects from blurry frames, we propose Impression Network that embodies a natural and efficient feature aggregation mechanism. In our framework, an impression feature is established by iteratively absorbing sparsely extracted frame features. The impression feature is propagated all the way down the video, enhancing features of low-quality frames. This impression mechanism makes it possible to perform long-range multi-frame feature fusion among sparse keyframes with minimum overhead. We demonstrate that Impression Network is significantly more accurate than per-frame detection baseline on ImageNet VID, while being 3 times faster (20 fps).