Microsoft COCO: Common Objects in Context

TLDR

The authors introduce the Microsoft COCO dataset to advance object recognition by contextualizing it within scene understanding and provide a detailed statistical comparison to PASCAL, ImageNet, and SUN. COCO comprises 328 k images of everyday scenes with 91 object categories, each annotated with per‑instance segmentations, totaling 2.5 million instances collected via crowd‑sourced interfaces. Baseline experiments with a Deformable Parts Model demonstrate the dataset’s utility for bounding‑box and segmentation detection.

Abstract

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.