Learning Attentive Pairwise Interaction for Fine-Grained Classification

TLDR

Fine‑grained classification is difficult because subtle differences among highly‑confused categories require discriminative representations, yet humans effectively use pairwise comparisons to identify contrastive clues. This paper introduces the Attentive Pairwise Interaction Network (API‑Net), which progressively recognizes fine‑grained image pairs through interaction. API‑Net first learns a mutual feature vector that captures semantic differences between two images, then uses it to generate gates for each image, allowing attentive contrastive clue extraction, and is trained end‑to‑end with a score‑ranking regularization across five benchmarks. API‑Net achieves state‑of‑the‑art accuracy on CUB‑200‑2011 (90.0%), Aircraft (93.9%), Stanford Cars (95.3%), Stanford Dogs (90.3%), and NABirds (88.1%).

Abstract

Fine-grained classification is a challenging problem, due to subtle differences among highly-confused categories. Most approaches address this difficulty by learning discriminative representation of individual input image. On the other hand, humans can effectively identify contrastive clues by comparing image pairs. Inspired by this fact, this paper proposes a simple but effective Attentive Pairwise Interaction Network (API-Net), which can progressively recognize a pair of fine-grained images by interaction. Specifically, API-Net first learns a mutual feature vector to capture semantic differences in the input pair. It then compares this mutual vector with individual vectors to generate gates for each input image. These distinct gate vectors inherit mutual context on semantic differences, which allow API-Net to attentively capture contrastive clues by pairwise interaction between two images. Additionally, we train API-Net in an end-to-end manner with a score ranking regularization, which can further generalize API-Net by taking feature priorities into account. We conduct extensive experiments on five popular benchmarks in fine-grained classification. API-Net outperforms the recent SOTA methods, i.e., CUB-200-2011 (90.0%), Aircraft (93.9%), Stanford Cars (95.3%), Stanford Dogs (90.3%), and NABirds (88.1%).