Ernie-Gram BiGRU Attention: An Improved Multi-Intention Recognition Model for Air Traffic Control

Abstract

In recent years, the emergence of large-scale pre-trained language models has made transfer learning possible in natural language processing, which overturns the traditional model architecture based on recurrent neural networks (RNN). In this study, we constructed a multi-intention recognition model, Ernie-Gram_Bidirectional Gate Recurrent Unit (BiGRU)_Attention (EBA), for air traffic control (ATC). Firstly, the Ernie-Gram pre-training model is used as the bottom layer of the overall architecture to implement the encoding of text information. The BiGRU module that follows is used for further feature extraction of the encoded information. Secondly, as keyword information is very important in Chinese radiotelephony communications, the attention layer after the BiGRU module is added to realize the extraction of keyword information. Finally, two fully connected layers (FC) are used for feature vector fusion and outputting intention classification vector, respectively. We experimentally compare the effects of two different tokenizer tools, the BERT tokenizer tool and Jieba tokenizer tool, on the final performance of the Bert model. The experimental results reveal that although the Jieba tokenizer tool has considered word information, the effect of the Jieba tokenizer tool is not as good as that of the BERT tokenizer tool. The final model’s accuracy is 98.2% in the intention recognition dataset of the ATC instructions, which is 2.7% higher than the Bert benchmark model and 0.7–3.1% higher than other improved models based on BERT.