Attention Mechanism Architecture for Arabic Sentiment Analysis

Abstract

This article tackles the problem of sentiment analysis in the Arabic language where a new deep learning model has been put forward. The proposed model uses a hybrid bidirectional gated recurrent unit (BiGRU) and bidirectional long short-term memory (BiLSTM) additive-attention model where the Bidirectional GRU/LSTM reads the individual sentence input from left to right and vice versa, enabling the capture of the contextual information. However, the model is trained on two types of embeddings: FastText and local learnable embeddings. The BiLSTM and BiGRU architectures are put into competition to identify the best hyperparameter set for the model. The developed model has been tested on three large-scale commonly employed Arabic sentiment dataset: large-scale Arabic Book Reviews Dataset (ABRD), Hotel Arabic-Reviews Dataset (HARD), and Books Reviews in the Arabic Dataset (BRAD). The testing results demonstrate that our model outperforms both the baseline models and the state-of-the-art models reported in the original references of these datasets, achieving accuracy scores of 98.6%, 96.19%, 95.65% for LARB, HARD, and BRAD, respectively. Furthermore, to demonstrate the generalization capabilities of our model, the performances of the model have been evaluated on three other natural language processing tasks: news categorization, offensive speech detection, and Russian sentiment analysis. The results demonstrated the developed model is language- and task-independent, which offers new perspectives for the application of the developed models in several other natural language processing challenges.