Multi-Oriented and Scale-Invariant License Plate Detection Based on Convolutional Neural Networks

Abstract

License plate detection (LPD) is the first and key step in license plate recognition.State-of-the-art object-detection algorithms based on deep learning provide a promising form ofLPD. However, there still exist two main challenges. First, existing methods often enclose objectswith horizontal rectangles. However, horizontal rectangles are not always suitable since licenseplates in images are multi-oriented, reflected by rotation and perspective distortion. Second, thescale of license plates often varies, leading to the difficulty of multi-scale detection. To addressthe aforementioned problems, we propose a novel method of multi-oriented and scale-invariantlicense plate detection (MOSI-LPD) based on convolutional neural networks. Our MOSI-LPD tightlyencloses the multi-oriented license plates with bounding parallelograms, regardless of the licenseplate scales. To obtain bounding parallelograms, we first parameterize the edge points of licenseplates by relative positions. Next, we design mapping functions between oriented regions andhorizontal proposals. Then, we enforce the symmetry constraints in the loss function and train themodel with a multi-task loss. Finally, we map region proposals to three edge points of a nearby licenseplate, and infer the fourth point to form bounding parallelograms. To achieve scale invariance, wefirst design anchor boxes based on inherent shapes of license plates. Next, we search different layersto generate region proposals with multiple scales. Finally, we up-sample the last layer and combineproposal features extracted from different layers to recognize true license plates. Experimental resultshave demonstrated that the proposed method outperforms existing approaches in terms of detectinglicense plates with different orientations and multiple scales.