基于BLSTM-CRF模型的安全漏洞领域命名实体识别

Named Entity Recognition for Vulnerabilities Based on BLSTM-CRF Model

摘要:

非结构化文本资源提供了大量与漏洞相关的信息，传统的特定领域实体识别依赖特征模板和领域知识来识别相关实体，其识别性能很大程度上依赖于人工选取的特征函数质量。如何利用机器挖掘文本隐含的特征，而不需要人工详细地制定领域术语的特征表达是一项具有挑战性的任务。该文针对安全漏洞领域，提出一种双向长短期记忆网络(Bidirectional Long Short-Term Memory，BLSTM)与条件随机场(Conditional Random Field，CRF)相结合的安全漏洞领域实体识别模型，并使用基于的词典方法对结果进行校正，F值可达到 85%以上。实验表明，该方法在提高实体识别的准确率和召回率的同时，能够显著地降低人工选取特征的工作量。

Abstract:

Unstructured text resources provide a large amount of information related to vulnerability. Traditional domain-specific entity recognition relies on feature templates and domain knowledge to recognize related entities. The recognition performance depends largely on the quality of manually selected feature functions. It is a challenging task to mine the features implied by the text automatically, rather than manually formulate the characterization of the domain terminology. In this paper, a BLSTM and CRF security vulnerability domain entity recognition model (BLSTM-CRF model) is proposed and a dictionary is used to correct the results generated by the model. The F value can reach 85%. Experiments show that this method can significantly reduce the workload of manually selecting features while improving the precision and recall