Multi-Orientation Scene Text Detection with Adaptive Clustering

Xu-Cheng Yin, Wei-Yi Pei, Jun Zhang, Hongwei Hao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2015 · 276 citations · 40 references

Concepts

TL;DR

Text detection in natural scene images is crucial for many content‑based image analysis tasks, yet most research focuses only on horizontal or near‑horizontal text. This work introduces a unified distance‑metric learning framework for adaptive hierarchical clustering and applies it to build a multi‑orientation scene text detection system. The system constructs text candidates through a coarse‑to‑fine grouping pipeline—morphology‑based single‑link clustering, orientation‑based divisive clustering, and projection‑based divisive clustering—using learned similarity weights and an automatically determined clustering threshold, and is evaluated on ICDAR 2011/2013, MSRA‑TD500, and NEOCR datasets while also releasing the USTB‑SV1K dataset. On MSRA‑TD500, the method achieves a 71 % f‑measure, surpassing state‑of‑the‑art results.

Abstract

Text detection in natural scene images is an important prerequisite for many content-based image analysis tasks, while most current research efforts only focus on horizontal or near horizontal scene text. In this paper, first we present a unified distance metric learning framework for adaptive hierarchical clustering, which can simultaneously learn similarity weights (to adaptively combine different feature similarities) and the clustering threshold (to automatically determine the number of clusters). Then, we propose an effective multi-orientation scene text detection system, which constructs text candidates by grouping characters based on this adaptive clustering. Our text candidates construction method consists of several sequential coarse-to-fine grouping steps: morphology-based grouping via single-link clustering, orientation-based grouping via divisive hierarchical clustering, and projection-based grouping also via divisive clustering. The effectiveness of our proposed system is evaluated on several public scene text databases, e.g., ICDAR Robust Reading Competition data sets (2011 and 2013), MSRA-TD500 and NEOCR. Specifically, on the multi-orientation text data set MSRA-TD500, the <inline-formula><tex-math>$f$</tex-math></inline-formula> measure of our system is <inline-formula><tex-math>$71$</tex-math></inline-formula> percent, much better than the state-of-the-art performance. We also construct and release a practical challenging multi-orientation scene text data set (USTB-SV1K), which is available at http://prir.ustb.edu.cn/TexStar/MOMV-text-detection/.

References

40