City-scale landmark identification on mobile devices

TLDR

With recent advances in mobile computing, the demand for visual localization or landmark identification on mobile devices is gaining interest. The study aims to improve city‑scale landmark identification by fusing facade‑aligned and viewpoint‑aligned street‑level image representations and providing a benchmark dataset to facilitate further research. The authors enhance feature detection in low‑contrast street‑level images and incorporate noisy GPS or network‑cell priors on user position, which prior methods often ignore. The fusion of facade‑aligned and viewpoint‑aligned representations significantly improves city‑scale recall rates, and the authors provide a repeatable evaluation scheme along with a publicly available dataset of 1.7 million images and challenging cell‑phone queries.

Abstract

With recent advances in mobile computing, the demand for visual localization or landmark identification on mobile devices is gaining interest. We advance the state of the art in this area by fusing two popular representations of street-level image data - facade-aligned and viewpoint-aligned - and show that they contain complementary information that can be exploited to significantly improve the recall rates on the city scale. We also improve feature detection in low contrast parts of the street-level data, and discuss how to incorporate priors on a user's position (e.g. given by noisy GPS readings or network cells), which previous approaches often ignore. Finally, and maybe most importantly, we present our results according to a carefully designed, repeatable evaluation scheme and make publicly available a set of 1.7 million images with ground truth labels, geotags, and calibration data, as well as a difficult set of cell phone query images. We provide these resources as a benchmark to facilitate further research in the area.