基于段落内部推理和联合问题答案匹配的选择型阅读理解模型

Reasoning over intra-document and jointly matching question and candidate answer to the passage based multiple-choice Reading Comprehension

摘要:

针对当前机器阅读理解方法中仅将问题与段落匹配会导致段落中的信息丢失或将问题和答案连接成单个序列与段落匹配会丢失问题与答案之间的交互，和传统的循环网络顺序解析文本从而忽略段落内部推理的问题，提出一种改进段落编码并且将段落与问题和答案共同匹配的模型.模型首先把段落在多个粒度下切分为块，编码器利用神经词袋表达将块内词嵌入向量求和，其次，将块序列通过前向全连接神经网络扩展到原始序列长度.然后，通过两层前向神经网络建模每个单词所在不同粒度的块之间的关系构造门控函数以使模型具有更大的上下文信息同时捕获段落内部推理.最后，通过注意力机制将段落表示与问题和答案的交互来选择答案.在SemEval-2018 Task 11任务上的实验结果表明，本文模型在正确率上超过了相比基线神经网络模型如Stanford AR 和GA Reader提高了9-10%，比最近的模型SurfaceLR至少提高了3%，超过TriAN的单模型1%左右.除此之外，在RACE数据集上的预训练也可以提高模型效果。

Abstract:

Accroding to the defects in the current machine reading comprehension approaches that either match the passage to the question alone leading to the loss of information of the passage or match the passage to the sequence that concatenates the question and the candidate answer leading to loss of interaction information between a question and an answer and traditional recurrent-based encoder that sequentially parses the text sequence ignoring inta-document relationships, a model that improve the encoder of the passage and jointly match the question and candidate answer to the passage is proposed. Firstly, the sequences of the passage are chunked into blocks based on multi-granular, encoder takes the neural bag-of-words representation of each block, that is sum the embedding of all words that reside in each block. Next, the blocks are passed into fully-connected layers and expanded to original sequence lengths. The gating function are then constructed through two layered feed-forward neural network which modeling the relationshiops between all blocks that each word resides in, allowing for possessing a larger overview of the context information and capturing the intra-document relationships. Finaly, the attention mechanism is used to model the interaction between the passage and the question as well as the candidate answer to select an answer. Experimental results on the SemEval-2018 Task 11 demonstrate that our approach performance improvement over the baselines such as Stanford AR and GA Reader ranges from %-%, pull ahead of recent model SurfaceLR by at least 3% and outperforms the TriAN by 1%. Besides, pretraining the model on RACE datasets helps to improve the overall performance.