Zero-Shot Object Counting

TLDR

Class‑agnostic object counting seeks to count arbitrary‑class instances at test time, but existing methods rely on human‑annotated exemplars that are often unavailable for novel categories, especially in autonomous systems. The authors introduce zero‑shot object counting, aiming to count objects using only the class name at test time by selecting optimal patches as counting exemplars. They construct a class prototype to identify class‑relevant patches, train a model to quantify patch suitability as counting exemplars, and then choose the most suitable patches for counting. Experimental results on the FSC‑147 class‑agnostic counting dataset validate the effectiveness of the proposed method. Code is available at https://github.com/cvlabstonybrook/zero-shot-counting.

Abstract

Class-agnostic object counting aims to count object instances of an arbitrary class at test time. Current methods for this challenging problem require human-annotated exemplars as inputs, which are often unavailable for novel categories, especially for autonomous systems. Thus, we propose zero-shot object counting (ZSC), a new setting where only the class name is available during test time. Such a counting system does not require human annotators in the loop and can operate automatically. Starting from a class name, we propose a method that can accurately identify the optimal patches which can then be used as counting exemplars. Specifically, we first construct a class prototype to select the patches that are likely to contain the objects of interest, namely class-relevant patches. Furthermore, we introduce a model that can quantitatively measure how suitable an arbitrary patch is as a counting exemplar. By applying this model to all the candidate patches, we can select the most suitable patches as exemplars for counting. Experimental results on a recent class-agnostic counting dataset, FSC-147, validate the effectiveness of our method. Code is available at https://github.com/cvlabstonybrook/zero-shot-counting.