EILPR: Toward End-to-End Irregular License Plate Recognition Based on Automatic Perspective Alignment

Abstract

Automatic License plate recognition (ALPR) remains a challenging task in face of some difficulties such as multi-line character distribution and license plate (LP) deformation due to camera angles. Most existing ALPR methods either focus on single-line LP or perform horizontal multi-line LP detection and recognition with character-level annotations. In this paper, we propose a novel end-to-end irregular license plate recognition (EILPR) to detect and recognize the LP of multi-line text or arbitrary shooting angles, using only plate-level annotations for training. In EILPR, a coarse-to-fine strategy is adopted to extract the LP features accurately for sequence recognition. Firstly, a coarse rectangular box of the LP is located, along with the corresponding predicted LP class which is single-line or double-line. Then, considering the fact that a LP mainly generates perspective distortion in the image due to its rigid feature, we propose a new automatic perspective alignment network (APAN) to extract the fine LP features connecting the detection and recognition. For recognition, a location-aware 2D attention based recognition network is performed to recognize the multi-line and multinational LP based on the extracted features. Experiments on several datasets show that EILPR achieves the state-of-the-art performance, demonstrating the effectiveness of the proposed method.