Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

TLDR

State‑of‑the‑art object detection networks rely on region proposal algorithms, and recent advances have reduced their runtime, exposing region proposal computation as a bottleneck. This work introduces a Region Proposal Network that shares full‑image convolutional features with the detection network, enabling nearly cost‑free region proposals. The RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position, is trained end‑to‑end, and is merged with Fast R‑CNN by sharing convolutional features so the unified network can attend to where to look. On VGG‑16, the system runs at 5 fps on a GPU while achieving state‑of‑the‑art accuracy on PASCAL VOC 2007/2012 and MS COCO with only 300 proposals, and it underpins first‑place entries in ILSVRC and COCO 2015 competitions.

Abstract

State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet and Fast R-CNN have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. We further merge RPN and Fast R-CNN into a single network by sharing their convolutional features---using the recently popular terminology of neural networks with 'attention' mechanisms, the RPN component tells the unified network where to look. For the very deep VGG-16 model, our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning entries in several tracks. Code has been made publicly available.