Annotating Object Instances with a Polygon-RNN

TLDR

While most current methods treat object segmentation as a pixel‑labeling problem, we cast it as a polygon‑prediction task that mirrors how datasets are annotated. The authors propose a semi‑automatic annotation approach that predicts polygon vertices sequentially, allowing annotators to intervene at any step to correct vertices. The method takes an image crop, outputs polygon vertices one by one, and lets human annotators correct vertices on the fly, producing accurate segmentations as desired. The approach speeds up annotation by 4.7× overall (7.3× for cars), achieves 78.4 % IoU agreement overall and 82.2 % for cars—matching human agreement—and generalizes to unseen datasets.

Abstract

We propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. In particular, our approach takes as input an image crop and sequentially produces vertices of the polygon outlining the object. This allows a human annotator to interfere at any time and correct a vertex if needed, producing as accurate segmentation as desired by the annotator. We show that our approach speeds up the annotation process by a factor of 4.7 across all classes in Cityscapes, while achieving 78:4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators. For cars, our speed-up factor is 7.3 for an agreement of 82:2%. We further show generalization capabilities of our approach to unseen datasets.