Document sentiment modeling based on topic attention hierarchy memory network

Abstract:

To the problem of prior knowledge and lack of semantic understanding in the traditional model of document level sentiment analysis, this paper proposes an sentiment analysis model called TWEANN（Attention Neural Networks based on Topicenhanced Word Embedding）, which is based on attention mechanism and hierarchical network feature representation. The word2vec model based on CBOW is used to train the word vector for corpus and the sparsity in the word vectors is reduced, the document topic distribution matrix is computed with LDA algorithm based on Gibbs sampling, the more complete text context information are obtained through hierarchical LSTM neural network and the deep sentiment features are finally extracted. The document topic distribution matrix is used as the model attention mechanism to extract the document features and the sentiment classification is thereby implemented. The experimental results show that the proposed TWEANN model has better classification results, compared with the TSA and HAN models. The F values on the Yelp2015, IMDB, and Amazon datasets is increased by 1.1%, 0.3%, and 1.8%, respectively, and the RMSE values on the Yelp2015 and Amazon datasets increased by 1.3% and 2.1%, respectively.