Sentiment and Context-Aware Hybrid DNN With Attention for Text Sentiment Classification

Abstract

A tremendous amount of unstructured data, such as comments, opinions, and other sorts of data is generated in real-time with the growth of web 2.0. Due to the unstructured nature of the data, building an accurate predictive model for sentiment analysis remains challenging. While various DNN architectures have been applied to sentiment analysis with encouraging results, they treat various features equally and suffer from high-dimensional feature space. Moreover, state-of-the-art methods cannot properly utilize semantic and sentiment knowledge to extract meaningful relevant contextual sentiment features. This paper proposes a sentiment and context-aware hybrid DNN model with an attention mechanism that intelligently learns and highlights salient features of relevant sentiment context in the text. We first use integrated wide coverage sentiment lexicons to identify text sentiment features then leverage bidirectional encoder representation from transformers to produce sentiment-enhanced word embeddings for text semantic extraction. After that, the proposed approach adapts the BiLSTM to capture both word order/contextual text semantic information and the long-dependency relation in the word sequence. Our model also employs an attention mechanism to assign weights to features and give greater significance to salient features in the word sequence. Finally, CNN is utilized to reduce the dimensionality of feature space and extract the local key features for sentiment analysis. The effectiveness of the proposed model is evaluated on real-world benchmark datasets demonstrating that the proposed model significantly improves the accuracy of existing text sentiment classification.