Sentiment Analysis on Text Reviews Using Lexicon Selected-Bert Embedding (LeBERT) Model with Convolutional Neural Network

Abstract

Abstract Sentiment analysis has become an important area of research in natural language processing. This technique has a wide range of applications such as comprehending user preferences in ecommerce feedback portals, politics, and in governance. However, accurate sentiment analysis requires robust text representation techniques that can convert words into precise vectors that represent the input text. There are two categories of text representation techniques: lexicon-based techniques and machine learning-based techniques. From research, both techniques have limitations. For instance, pre-trained word embeddings such as Word2Vec, Glove and Bidirectional Encoder Representations from Transformers (BERT) generate vectors by considering word distances, similarities and occurrences ignoring other aspects such as word sentiment orientation. Aiming at such limitations, this paper presents a sentiment classification model (named LeBERT) combining Sentiment Lexicon, N-grams, BERT and CNN. In the model, Sentiment Lexicon, N-grams and BERT are used to vectorize words selected from a section of the input text. CNN is used as the deep neural network classifier for feature mapping and giving the output sentiment class. The proposed model is evaluated on Yelp’s three datasets (movie, restaurant and products’ reviews) using accuracy, precision and F-measure as performance metrics. The experimental results indicate that the proposed LeBERT model outperform the existing state-of-the-art models with an F-measure score of 88.73% in binary sentiment classification.