Research on the Construction of a Bidirectional Neural Network Machine Translation Model Fused with Attention Mechanism

Abstract

With the development of deep learning, neural machine translation has also been paid attention and developed by researchers. Especially in the application of encoder-decoder in natural language processing, the translation performance has been significantly improved. In 2014, the attention mechanism was used in neural machine translation, the performance of translation was greatly improved, and the interpretability of the model was increased. This research proposes a research idea of sparsemax combined with AAN machine translation model and conducts multiple ablation experiments for experimental verification. This chapter first studies the problem of insufficient sparse normalization when generating target words in the attention mechanism and studies the neural machine translation model incorporating the sparse normalization calculation method. It solves the problem of inductive bias in the data transfer process of related sub-layers in the model. By combining the strategy of sparse normalization, the similarity value of related word vectors can be obtained more accurately when aligning words, which is more convenient for this chapter. Calculate and analyze the specific principles of the model. In addition, when the model faces a large vocabulary in the decoding stage, too many weights of scattered vocabulary vectors are not conducive to the generation of correct target values. After using the sparse normalization strategy, it can reduce the number of inconveniences. The calculation between related words optimizes the classification accuracy of the target vocabulary. In this chapter, aiming at the waste of the transformer’s decoder calculation in the inference stage, the average attention structure is used to replace the attention calculation layer of the first layer of the decoder part of the original model. Each moment is only related to the previous moment, which alleviates the waste of computing resources.