MatchNet: Unifying feature and metric learning for patch-based matching

TLDR

The authors propose a unified approach that jointly learns patch feature representations and comparison functions for a patch matching system. MatchNet is a deep convolutional network that extracts patch features, followed by a three‑layer fully connected similarity module, trained on standard datasets with synthetic exemplar pairs to reduce overfitting and evaluated through comprehensive experiments. The unified model achieves higher accuracy than prior state‑of‑the‑art patch matchers, reduces descriptor storage, and improves matching efficiency by separating feature extraction and similarity computation, with pre‑trained weights released publicly.

Abstract

Motivated by recent successes on learning feature representations and on learning feature comparison functions, we propose a unified approach to combining both for training a patch matching system. Our system, dubbed Match-Net, consists of a deep convolutional network that extracts features from patches and a network of three fully connected layers that computes a similarity between the extracted features. To ensure experimental repeatability, we train MatchNet on standard datasets and employ an input sampler to augment the training set with synthetic exemplar pairs that reduce overfitting. Once trained, we achieve better computational efficiency during matching by disassembling MatchNet and separately applying the feature computation and similarity networks in two sequential stages. We perform a comprehensive set of experiments on standard datasets to carefully study the contributions of each aspect of MatchNet, with direct comparisons to established methods. Our results confirm that our unified approach improves accuracy over previous state-of-the-art results on patch matching datasets, while reducing the storage requirement for descriptors. We make pre-trained MatchNet publicly available.