基于主题注意力层次记忆网络的文档情感建模

Document sentiment modeling based on topic attention hierarchy memory network

摘要:

针对文档水平情感分析传统模型存在先验知识依赖以及语义理解不足问题,提出一种基于注意力机制与层次网络特征表示的情感分析模型TWEANN.采用基于CBOW方式的word2vec模型针对语料训练词向量,减小词向量间的稀疏度,使用基于Gibbs采样的LDA算法计算出文档主题分布矩阵,继而通过层次LSTM神经网络获取更为完整的文本上下文信息从而提取出深度情感特征,将文档主题分布矩阵作为模型注意力机制提取文档特征,从而实现情感分类.实验结果表明:提出的TWEANN模型较TSA、HAN模型分类效果较好,在Yelp2015、IMDB、Amazon数据集上的F值分别提升了1.1%、0.3%、1.8%,在Yelp2015和Amazon数据集上的RMSE值分别提升了1.3%、2.1%.

Abstract:

To the problem of prior knowledge and lack of semantic understanding in the traditional model of document level sentiment analysis, this paper proposes an sentiment analysis model called TWEANN（Attention Neural Networks based on Topicenhanced Word Embedding）, which is based on attention mechanism and hierarchical network feature representation. The word2vec model based on CBOW is used to train the word vector for corpus and the sparsity in the word vectors is reduced, the document topic distribution matrix is computed with LDA algorithm based on Gibbs sampling, the more complete text context information are obtained through hierarchical LSTM neural network and the deep sentiment features are finally extracted. The document topic distribution matrix is used as the model attention mechanism to extract the document features and the sentiment classification is thereby implemented. The experimental results show that the proposed TWEANN model has better classification results, compared with the TSA and HAN models. The F values on the Yelp2015, IMDB, and Amazon datasets is increased by 1.1%, 0.3%, and 1.8%, respectively, and the RMSE values on the Yelp2015 and Amazon datasets increased by 1.3% and 2.1%, respectively.