Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

TLDR

The spatial pyramid framework provides insights into the effectiveness of recent image descriptors such as Torralba's gist and Lowe's SIFT. The paper proposes a method for recognizing scene categories using approximate global geometric correspondence. The method partitions images into increasingly fine sub‑regions, computes histograms of local features within each, and aggregates them to form a spatial pyramid representation. The spatial pyramid yields a simple, computationally efficient extension of bag‑of‑features that significantly outperforms prior approaches, achieving state‑of‑the‑art accuracy on Caltech‑101 and high accuracy on a fifteen‑category natural scene database.

Abstract

This paper presents a method for recognizing scene categories based on approximate global geometric correspondence. This technique works by partitioning the image into increasingly fine sub-regions and computing histograms of local features found inside each sub-region. The resulting "spatial pyramid" is a simple and computationally efficient extension of an orderless bag-of-features image representation, and it shows significantly improved performance on challenging scene categorization tasks. Specifically, our proposed method exceeds the state of the art on the Caltech-101 database and achieves high accuracy on a large database of fifteen natural scene categories. The spatial pyramid framework also offers insights into the success of several recently proposed image descriptions, including Torralba's "gist" and Lowe's SIFT descriptors.