Shape Context: A New Descriptor for Shape Matching and Object Recognition

TLDR

The key problem is finding pointwise correspondences between an image shape and a stored prototype shape. The authors develop an object recognition approach that uses a new shape descriptor, the shape context, to match shapes and classify them with a nearest‑neighbor similarity measure. Shape contexts capture the distribution of relative positions of shape points, and after alignment they provide a robust similarity score used for classification. Shape contexts simplify point correspondences and enable a nearest‑neighbor classifier that achieves a 0.63% error rate on MNIST, outperforming prior methods.

Abstract

We develop an approach to object recognition based on matching shapes and using a resulting measure of similarity in a nearest neighbor classifier. The key algorithmic problem here is that of finding pointwise correspondences between an image shape and a stored prototype shape. We introduce a new shape descriptor, the shape context, which makes this possible, using a simple and robust algorithm. The shape context at a point captures the distribution over relative positions of other shape points and thus summarizes global shape in a rich, local descriptor. We demonstrate that shape contexts greatly simplify recovery of correspondences between points of two given shapes. Once shapes are aligned, shape contexts are used to define a robust score for measuring shape similarity. We have used this score in a nearest-neighbor classifier for recognition of hand written digits as well as 3D objects, using exactly the same distance function. On the benchmark MNIST dataset of handwritten digits, this yields an error rate of 0.63%, outperforming other published techniques.