Combining BERT with TCN-BiGRU for enhancing Arabic aspect category detection

Abstract

Aspect-based sentiment analysis (ABSA) is a challenging task of sentiment analysis that aims at extracting the discussed aspects and identifying the sentiment corresponding to each aspect. We can distinguish three main ABSA tasks: aspect term extraction, aspect category detection (ACD), and aspect sentiment classification. Most Arabic ABSA research has relied on rule-based or machine learning-based methods, with little attention to deep learning techniques. Moreover, most existing Arabic deep learning models are initialized using context-free word embedding models, which cannot handle polysemy. Therefore, this paper aims at overcoming the limitations mentioned above by exploiting the contextualized embeddings from pre-trained language models, specifically the BERT model. Besides, we combine BERT with a temporal convolutional network and a bidirectional gated recurrent unit network in order to enhance the extracted semantic and contextual features. The evaluation results show that the proposed method has outperformed the baseline and other models by achieving an F1-score of 84.58% for the Arabic ACD task. Furthermore, a set of methods are examined to handle the class imbalance in the used dataset. Data augmentation based on back-translation has shown its effectiveness through enhancing the first results by an overall improvement of more than 3% in terms of F1-score.