An Improved Self Attention Mechanism Based on Optimized BERT-BiLSTM Model for Accurate Polarity Prediction

Abstract

Abstract Polarity prediction is the field of study that discovers people’s opinions, feelings, assessments, perspectives and feelings about associations and their attributes as communicated in written text. It is one of the most active research areas in the field of text mining. Nowadays online reviews play an important role by giving a helping hand to the customers to know about other customer’s opinions about the product they are going to purchase. This also guides the organizations and government sectors to increase their quality of product and services. Pre-trained BERT (Bidirectional Encoder Representations from Transformers) is used for word embedding in this model. The fine-tuned BERT is used for better word representation which in turn improves the sentimental analysis classification accuracy. Bidirectional Long Short-Term Memory classifier is utilized for polarity prediction. To enhance the performance of Bidirectional Long Short-Term Memory, the weight parameters of Bi-directional LSTM are optimally selected by using APSO algorithm. Improved self-attention mechanism is added with BiLSTM for focusing on significant words in the context. For performance analysis, four bench mark datasets are used for experiments.