SUN database: Large-scale scene recognition from abbey to zoo

TLDR

Scene categorization is a fundamental problem in computer vision, yet existing datasets are limited, with the largest scene database containing only 15 categories. This work introduces the SUN database, comprising 899 scene categories and 130,519 images. The authors evaluate state‑of‑the‑art scene recognition algorithms on 397 well‑sampled SUN categories, compare them to human performance, and investigate finer‑grained scene representations within larger scenes. These evaluations establish new performance bounds for scene recognition algorithms on the SUN database.

Abstract

Scene categorization is a fundamental problem in computer vision. However, scene understanding research has been constrained by the limited scope of currently-used databases which do not capture the full variety of scene categories. Whereas standard databases for object categorization contain hundreds of different classes of objects, the largest available dataset of scene categories contains only 15 classes. In this paper we propose the extensive Scene UNderstanding (SUN) database that contains 899 categories and 130,519 images. We use 397 well-sampled categories to evaluate numerous state-of-the-art algorithms for scene recognition and establish new bounds of performance. We measure human scene classification performance on the SUN database and compare this with computational methods. Additionally, we study a finer-grained scene representation to detect scenes embedded inside of larger scenes.