Spatial Transform Depthwise Over-Parameterized Convolution Recurrent Neural Network for License Plate Recognition in Complex Environment

Abstract

Abstract Automatic license plate recognition (ALPR) system has been widely used in intelligent transportation and other fields. However, in complex environments such as vehicle sound source localization, poor illumination, or bad weather conditions, ALPR is still a challenging problem. Aiming at the problem, an end-to-end deep learning framework is developed based on depthwise over-parameterized convolution recurrent neural network for license plate character recognition. The proposed framework is composed as follows: (i) license plate correcting module based on spatial transformation network; (ii) feature extraction module based on depthwise over-parameterized convolution; (iii) sequence annotation module based on bidirectional long short-term memory; and (iv) regularized sequence decoding module based on connectionist temporal classification with maximum conditional entropy. Two open-source datasets of Chinese License Plate Datasets (SYSU) and Chinese City Parking Dataset (CCPD) are used to verify the performance of the algorithm. The proposed end-to-end framework can effectively rectify distorted and inclined license plates in spatial domain. It can recognize license plates without complex character segmentation process. Compared with some current state-of-art algorithms, the proposed algorithm achieved the best performance with the recognition accuracy of 96.31% and 88.31% based on the two datasets of SYSU and CCPD, respectively.