A Deep CRNN-Based Sentiment Analysis System with Hybrid BERT Embedding

Abstract

This paper proposes a novel hybrid embedding to enhance scope of word embeddings by augmenting these with natural language processing operations. We primarily focus on the proposal of new hybrid word embedding generated by augmenting BERT embedding vectors with polarity score. The paper further proposes a new deep learning architecture inspired by the use of convolutional neural network for feature extraction and a bidirectional recurrent network for contextual and temporal feature exploitation. Use of CNN with hybrid embedding allowed the network to extract even the higher-level styles in writing, while bidirectional RNN helped in understanding context. The paper justifies that the proposed architecture and hybrid embedding improves performance of sentiment classification system by performing a large number of experiments and testing on a number of deep learning architectures. The architecture on new hybrid embeddings incurred an accuracy of 96%, which is a significant improvement when compared with recent studies in the literature.