基于残差空洞卷积神经网络的网络安全实体识别方法

doi:10.11959/j.issn.2096-109x.2020009

Abstract

Abstract:

In recent years,cybersecurity threats have increased,and data-driven security intelligence analysis has become a hot research topic in the field of cybersecurity.In particular,the artificial intelligence technology represented by the knowledge graph can provide support for complex cyberattack detection and unknown cyberattack detection in multi-source heterogeneous threat intelligence data.Cybersecurity entity recognition is the basis for the construction of threat intelligence knowledge graphs.The composition of security entities in open network text data is very complex,which makes traditional deep learning methods difficult to identify accurately.Based on the pre-training language model of BERT (pre-training of deep bidirectional transformers),a cybersecurity entity recognition model BERT-RDCNN-CRF based on residual dilation convolutional neural network and conditional random field was proposed.The BERT model was used to train the character-level feature vector representation.Combining the residual convolution and the dilation neural network model to effectively extract the important features of the security entity,and finally obtain the BIO annotation of each character through CRF.Experiments on the large-scale cybersecurity entity annotation dataset constructed show that the proposed method achieves better results than the LSTM-CRF model,the BiLSTM-CRF model and the traditional entity recognition model.

Key words: cybersecurity,entity recognition, residual connection, dilation convolution neural network, BERT pre-train model

CLC Number:

TP391

Bo XIE,Guowei SHEN,Chun GUO,Yan ZHOU,Miao YU. Cyber security entity recognition method based on residual dilation convolution neural network[J]. Chinese Journal of Network and Information Security, 2020, 6(5): 126-138.

Figures/Tables 11

References 28

[1]	SHU X , ARAUJO F , SCHALES D L ,et al. Threat intelligence computing[C]// ACM SIGSAC Conference on Computer and Communications Security. 2018: 1883-1898.
[2]	王通, 艾中良, 张先国 . 基于深度学习的威胁情报知识图谱构建技术[J]. 计算机与现代化, 2018(12): 21-26.
	WANG T , AI Z L , ZHANG X G . Knowledge graph construction of threat intelligence based on deep learning[J]. Computer and Modernization, 2018(12): 21-26.
[3]	刘浏, 王东波 . 命名实体识别研究综述[J]. 情报学报, 2018(3): 329-340.
	LIU L.WANG D B . A review on named entity recognition[J]. Journal of the China Society for Scientific and Technical Information, 2018(3): 329-340.
[4]	张晓斌, 陈福才, 黄瑞阳 . 基于CNN和双向LSTM融合的实体关系抽取[J]. 网络与信息安全学报, 2018,4(9): 44-51.
	ZHANG X B , CHEN F C , HUANG R Y . Relation extraction based on CNN and Bi-LSTM[J]. Chinese Journal of Network and Information Security, 2018,4(9): 44-51.
[5]	GEORGESCU T M , IANCU B , ZURINI M . Named-entityrecognition-based automated system for diagnosing cybersecurity situations in IoT networks[J]. Sensors, 2019,19(15):3380.
[6]	HAMMERTON J , . Named entity recognition with long short-term memory[C]// The 7th Conference on Natural Language Learning at Hltnaacl. 2003: 172-175.
[7]	COLLOBERT R , WESTON J , BOTTOU L ,et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011,12(8): 2493-2537.
[8]	SANTOS C N , GUIMARAES V . Boosting named entity recognition with neural character embeddings[J]. arXiv preprint arXiv:1505.05008, 2015
[9]	CHIU J P C , NICHOLS E . Named entity recognition with bidirectional LSTM-CNNs[J]. Transactions of the Association for Computational Linguistics, 2016,4: 357-370.
[10]	STRUBELL E , VERGA P , BELANGER D ,et al. Fast and accurate entity recognition with iterated dilated convolutions[J]. arXiv preprint arXiv:1702.02098, 2017
[11]	HE J , WANG H . Chinese named entity recognition and word segmentation based on character[C]// The Sixth SIGHAN Workshop on Chinese Language Processing. 2008.
[12]	LIU Z , ZHU C , ZHAO T . Chinese named entity recognition with a sequence labeling approach:based on characters,or based on words[C]// International Conference on Intelligent Computing. 2010: 634-640.
[13]	LI H , HAGIWARA M , LI Q ,et al. Comparison of the impact of word segmentation on name tagging for Chinese and Japanese[C]// LREC. 2014: 2532-2536.
[14]	秦娅, 申国伟, 赵文波 ,等. 基于深度神经网络的网络安全实体识别方法[J]. 南京大学学报(自然科学), 2019,55(1): 29-40.
	QIN Y , SHEN G W , ZHAO W B ,et al. Research on the method of network security entity recognition based on deep neural network[J]. Journal of Nanjing University (Natural Sciences), 2019,55(1): 29-40.
[15]	XU Y , WANG Y , LIU T ,et al. Joint segmentation and named entity recognition using dual decomposition in Chinese discharge summaries[J]. Journal of the American Medical Informatics Association, 2013,21(e1):e84-e92.
[16]	ZHANG Y , YANG J . Chinese NER using lattice LSTM[J]. arXiv preprint arXiv:1805.02023, 2018
[17]	MNIH V , HEESS N , GRAVES A . Recurrent models of visual attention[C]// Advances in Neural Information Processing Systems 27 (NIPS 2014). 2014: 2204-2212.
[18]	BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014
[19]	YIN W , SCHüTZE H , XIANG B ,et al. AbCNN:attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the Association for Computational Linguistics, 2016,4: 259-272.
[20]	WANG L , CAO Z , DE-MELO G ,et al. Relation classification via multi-level attention CNNs[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 1298-1307.
[21]	DEVLIN J , CHANG M W , LEE K ,et al. Bert:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018
[22]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[23]	LAFFERTY J , MCCALLUM0 -A , PEREIRA F C N . Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]// Proceedings of the Eighteenth International Conference on Machine Learning. 2001: 282-289.
[24]	HOCHREITER S , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[25]	PENG N , DREDZE M . Named entity recognition for chinese social media with jointly trained embeddings[C]// 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 548-554.
[26]	LAMPLE G , BALLESTEROS M , SUBRAMANIAN S ,et al. Neural architectures for named entity recognition[J]. arXiv preprint arXiv:1603.01360, 2016
[27]	MA X , HOVY E . End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF[J]. arXiv preprint arXiv:1603.01354, 2016
[28]	ZEILER M D . ADADELTA:an adaptive learning rate method[J]. arXiv preprint arXiv:1212.5701, 2012

Metrics

Recommended 0

No Suggested Reading articles found!

实体类别	训练集	验证集	测试集	总计
PER	8 737	1 256	2 570	12 563
LOC	16 454	2 372	4 890	23 716
ORG	12 755	1 813	3 566	18 134
SW	4 649	673	1 407	6 729
RT	56 190	8 045	16 212	80 447
VUL_ID	4 649	673	1 407	6 729
总计	103 434	14 832	30 052	148 318

参数名	参数描述	参数值
h	窗口大小	3
n	特征图数量	128
p	Dropout率	0.5
b	批处理大小	64
δ	卷积空洞率	2
n_r	残差块数量	4

模型	准确率	精确率	召回率	F1值
CRF	0.915 0	0.842 6	0.733 4	0.784 2
LSTM	0.923 6	0.837 5	0.806 2	0.821 6
LSTM-CRF	0.929 5	0.861 7	0.820 7	0.840 7
BiLSTM-CRF	0.928 3	0.847 0	0.851 8	0.849 4
CNN-BiLSTM-CRF	0.931 0	0.864 7	0.840 7	0.852 5
FT-CNN-BiLSTM-CRF	0.933 1	0.884 5	0.836 8	0.860 0
BERT-CRF	0.959 4	0.824 7	0.809 5	0.817 1
BERT-LSTM-CRF	0.975 3	0.883 0	0.919 6	0.901 0
BERT-BiLSTM-CRF	0.973 4	0.846 9	0.903 1	0.874 1
BERT-GRU-CRF	0.976 4	0.884 3	0.913 0	0.898 4
BERT-BiGRU-CRF	0.974 1	0.826 7	0.902 5	0.862 9
BERT-RDCNN-CRF	0.976 8	0.887 2	0.910 7	0.898 8

序号	例句	BERT-RDCNN-Attention-CRF	真实标签
1	Wardle 在某篇博文中表示有了 RansomWhere,RansomWhere在被发现和阻止前，理想情况下最多只能加密几个文件	RT：加密，文件SW：RansomWhere PER：Wardle	RT：加密，文件SW：RansomWhere PER：Wardle
2	早在 5 年多以前，SemiAccurate 专家 Charlie， Demerjian在研究硬件后门时就知晓了该漏洞	ORG：SemiAccurate PER：Charlie，Demerjian RT：硬件，后面，漏洞	PER：Charlie，Demerjian RT：硬件，后面，漏洞
3	这种方案是 FIDO 联盟提出来的，苹果的芯片和Android手机芯片基本遵循这套方案	ORG：FIDO联盟，苹果，Android RT：芯片	ORG：苹果SW：Android RT：芯片
4	需要通过逆向工程来找到Sign nature的位置	RT：逆向工程	RT：逆向工程
5	将加密后的IP地址去除首尾的分隔符经过base64解密并于036异或之后得到真实的IP地址在本样本中	RT：IP地址	RT：IP地址
6	暗云Ⅲ木马专杀工具	SW：暗云ⅢRT：木马	SW：暗云ⅢRT：木马
7	当然，选择OpenResty也可以，如果选择OpenResty就不需要单独安装lua相关的组件	SW：OpenResty	无
8	编号 CVE-2017-0882 的漏洞可导致拥有向其他用户发送issue或merge请求权限的攻击者获取到该用户的信息	VUL_ID：CVE-2017-0882 RT：漏洞，用户，请求，权限，攻击者	VUL_ID：CVE-2017-0882 RT：漏洞，用户，请求，权限，攻击者
9	除了CVE-2017-0199之外，漏洞受欢迎程度排名第二位和第三位的分别是CVE-2012-0158和CVE-2015-1641	VUL_ID：CVE-2017-0199，CVE-20120158，CVE-2015-1641	VUL_ID：CVE-2017-0199，CVE2012 - 0158，CVE-2015-1641

Cyber security entity recognition method based on residual dilation convolution neural network

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 28

Related Articles 15

Metrics

Recommended 0

[1]	Saite CHEN, Weihai LI, Yuanzhi YAO, Nenghai YU. Location privacy protection method based on lightweight K-anonymity incremental nearest neighbor algorithm [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 60-72.
[2]	Wanze CHEN, Liqing HUANG, Jiazhen CHEN, Feng YE, Tianqiang HUANG, Haifeng LUO. Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 150-160.
[3]	Heli WANG, Qiao YAN. Selfish mining detection scheme based on the characters of transactions [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 104-114.
[4]	Xiaochen SHEN, Yinhui GE, Bo CHEN, Ling YU. Research on construction technology of artificial intelligence security knowledge graph [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 164-174.
[5]	Jingwen LI, Yawen LI. Application and risk response of deep synthesis technology [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 184-190.
[6]	Long DAI, Jing ZHANG, Xuefeng FAN, Xiaoyi ZHOU. NLP neural network copyright protection based on black box watermark [J]. Chinese Journal of Network and Information Security, 2023, 9(1): 140-149.
[7]	Honghao ZHENG, Yinuo HAO, Hongtao YU, Shaomei LI, Yiteng WU. Rumor detection in social media based on eahanced Transformer [J]. Chinese Journal of Network and Information Security, 2022, 8(4): 168-174.
[8]	Yu ZHANG, Binglong LI, Xuejuan LI, Heyu ZHANG. Evidence classification method of chat text based on DSR and BGRU model [J]. Chinese Journal of Network and Information Security, 2022, 8(2): 150-159.
[9]	Xiangdong HU, Zhengguo TIAN. Methods of security situation prediction for industrial internet fused attention mechanism and BSRU [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 41-51.
[10]	Pengcheng WANG, Haibin ZHENG, Jianfei ZOU, Ling PANG, Hu LI, Jinyin CHEN. Robustness evaluation of commercial liveness detection platform [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 180-189.
[11]	Yaofei WANG, Weiming ZHANG, Kejiang CHEN, Wenbo ZHOU, Nenghai YU. Survey on image non-additive steganography [J]. Chinese Journal of Network and Information Security, 2021, 7(6): 1-10.
[12]	Jie QIU, Rui HAN, Zhifeng WEI, Zhiyang WANG. Research of public infrastructure system and security policy in cyberspace [J]. Chinese Journal of Network and Information Security, 2021, 7(6): 56-67.
[13]	Yuxiang CHENG, Weiming ZHANG, Weixiang LI, Nenghai YU. Binary image steganography method based on layered embedding [J]. Chinese Journal of Network and Information Security, 2021, 7(5): 49-56.
[14]	Honghao ZHENG, Hongtao YU, Shaomei LI. Chinese NER based on improved Transformer encoder [J]. Chinese Journal of Network and Information Security, 2021, 7(5): 105-112.
[15]	Xiao YANG,Xiuzhen CHEN,Jin MA,Haozhe LIANG,Shenghong LI. User interests-based microblog tracing algorithm [J]. Chinese Journal of Network and Information Security, 2020, 6(6): 164-173.