基于卷积神经网络和自注意力机制的文本分类模型

Text classification model based on convolutional neural network and selfattention mechanism

摘要:

摘 要： 单词级别的浅层卷积神经网络（CNN）模型在文本分类任务上取得了良好的表现.然而，浅层CNN模型由于无法捕捉长距离依赖关系，影响了模型的在文本分类任务上的效果.简单地加深模型层数并不能提升模型的效果.本文提出一种新的单词级别的文本分类模型Word-CNN-Att，该模型使用CNN捕捉局部特征和位置信息，利用自注意力机制捕捉长距离依赖.在AGNews、DBPedia、Yelp Review Polarity、Yelp Review Full、Yahoo! Answers 5个公开的数据集上，Word-CNN-Att比单词级别的浅层CNN模型的准确率分别提高了0.9%、0.2%、0.5%、2.1%、2.0%.

Abstract:

The wordlevel shallow convolutional neural network (CNN) model has achieved good performance in text classification tasks. However, shallow CNN models cant capture longrange dependencies, which affects the model's performance in text classification tasks, but simply deepening the number of layers of the model does not improve the models performance. This paper proposes a new wordlevel text classification model WordCNNAtt, which uses CNN to capture local features and position information, and captures longrange dependencies with selfattention mechanism. The accuracy of WordCNNAtt, on the five public datasets of AGNews, DBPedia, Yelp Review Polarity, Yelp Review Full, Yahoo! Answers, is 0.9%, 0.2%, 0.5%, 2.1%, and 2.0% higher than the wordlevel shallow CNN model respectively.