基于CNN和双向LSTM融合的实体关系抽取

doi:10.11959/j.issn.2096-109x.2018074

Abstract

Abstract:

Relation extraction aims to identify the entities in the Web text and extract the implicit relationships between entities in the text.Studies have shown that deep neural networks are feasible for relation extraction tasks and are superior to traditional methods.Most of the current relation extraction methods apply convolutional neural network (CNN) and long short-term memory neural network (LSTM) methods.However,CNN just considers the correlation between consecutive words and ignores the correlation between discontinuous words.On the other side,although LSTM takes correlation between long-distance words into account,the extraction features are not sufficiently extracted.In order to solve these problems,a relation extraction method that combining CNN and LSTM was proposed.three methods were used to carry out the experiments,and confirmed the effectiveness of these methods,which had some improvement in F1 score.

Key words: relation extraction, convolution neural networks, long short-term memory, attention mechanism

CLC Number:

TP393

Xiaobin ZHANG, Fucai CHEN, Ruiyang HUANG. Relation extraction based on CNN and Bi-LSTM[J]. Chinese Journal of Network and Information Security, 2018, 4(9): 44-51.

Figures/Tables 10

References 16

[1]	IRIS H , . Semeval-2010 task 8:multi-way classification of semantic relations between pairs of nominals[C]// The Workshop on Semantic Evaluations:Recent Achievements and Future Directions. 2009.
[2]	车万翔, 刘挺, 李生 . 实体关系自动抽取[J]. 中文信息学报, 2005,19(2): 2-7.
	CHE W X , LIU T , LI S . Automatic entity relation extraction[J]. Journal of Chinese Information Processing, 2005,19(2): 2-7.
[3]	黄瑞红, 孙乐, 冯元勇, 黄云平 . 基于核方法的中文实体关系抽取研究[J]. 中文信息学报, 2008,22(5): 102-108.
	HUANG R H , SUN L , FENG Y Y ,et al. A study on kernel based chinese relation extraction[J]. Jornal of Chinese Information Processing, 2008,22(5): 102-108.
[4]	LIU C Y , SUN W B , CHAO W H ,et al. Convolution neural network for relation extraction[C]// International Conference on Advanced Data Mining and Applications. 2013.
[5]	ZENG D J ,et al. Relation classification via convolutional deep neural network[C]// COLING. 2014.
[6]	ZENG D J , LIU K , CHEN Y B ,et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]// The 2015 Conference on Empirical Methods in Natural Language Processing. 2015.
[7]	ZHANG D X , WANG D . Relation classification via recurrent neural network[J]. arXiv preprint arXiv:1508.01006, 2015.
[8]	ZHANG S , ZHENG D Q , HU X C ,et al. bidirectional long short-term memory networks for relation classification[C]// PACLIC. 2015.
[9]	ZHOU P , et al . Attention-based bidirectional long short-term memory networks for relation classification[C]// The 54th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers). 2016.
[10]	TOMAS M , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. arXiv:1301.3781, 2013.
[11]	SEPP H , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[12]	NITISH S . Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014,15(1): 1929-1958.
[13]	DIEDERIK K P , BA J . Adam:a method for stochastic optimization[J]. arXiv:1412.6980, 2014.
[14]	YAN X ,et al. classifying relations via long short term memory networks along shortest dependency paths[C]// EMNLP. 2015.
[15]	WANG L L , ZHU C , et al . Relation classification via multi-level attention cnns[C]// The 54th Annual Meeting of the Association for Computational Linguistics. 2016.
[16]	BRYAN R , HARABAGIU S . Utd:classifying semantic relations by combining lexical and semantic resources[C]// The 5th International Workshop on Semantic Evaluation. 2010.

Metrics

Recommended 0

No Suggested Reading articles found!

实验环境	配置
操作系统	Win 10 64位
CPU	Intel i7-6700HQ@2.6GHz (8CPU)
GPU	NVIDIA GeForce GTX 1060
内存	16 GB
编程语言	Python 3.6.3
深度学习框架	TensorFlow 1.5.0

关系类别	个数（比例）
Other	1 410 (17.63%)
Cause-Effect	1 003 (12.54%)
Component-Whole	941 (11.76%)
Entity-Destination	845 (10.56%)
Product-Producer	717 ( 8.96%)
Entity-Origin	716 ( 8.95%)
Member-Collection	690 ( 8.63%)
Message-Topic	634 ( 7.92%)
Content-Container	540 ( 6.75%)
Instrument-Agency	504 ( 6.30%)

参数	值
词向量维度	300
位置向量维度	20
词性标注向量维度	60
批处理大小	128
adam学习率	1.00×10^?3
LSTM层dropout	0.3
其他dropout	0.5
L2正则化系数	1.00×10^?4
LSTM单元大小	512
CNN卷积核窗口长度	3
CNN卷积核个数	512

模型	准确率	召回率	F1值
SVM^[16]	82.25%	82.28%	82.19
PCNN	78.08%	87.26%	82.34
BLSTM	79.28%	82.61%	80.75
PCNN_BLSTM	82.32%	87.14%	84.46
BLSTM_PCNN	78.93%	84.73%	81.32
PCNN+BLSTM	81.25%	86.64%	83.78

Relation extraction based on CNN and Bi-LSTM

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 16

Related Articles 7

Metrics

Recommended 0

[1]	Dengyong ZHANG, Huang WEN, Feng LI, Peng CAO, Lingyun XIANG, Gaobo YANG, Xiangling DING. Image inpainting forensics method based on dual branch network [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 110-122.
[2]	Fan GAO, Jian WANG, Jiqiang LIU. Research on link detection technology based on dynamic browser fingerprint [J]. Chinese Journal of Network and Information Security, 2022, 8(4): 144-156.
[3]	Shize GUO, Fan ZHANG, Zhuoxue SONG, Ziming ZHAO, Xinjie ZHAO, Xiaojuan WANG, Xiangyang LUO. Detection of SSL/TLS protocol attacks based on flow spectrum theory [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 30-40.
[4]	Xiangdong HU, Zhengguo TIAN. Methods of security situation prediction for industrial internet fused attention mechanism and BSRU [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 41-51.
[5]	Jian JIA,Linfeng LIU,Jiagao WU. Charging pile recommendation method for idle electric taxis based on recurrent neural network [J]. Chinese Journal of Network and Information Security, 2020, 6(6): 152-163.
[6]	Yang SUN,Chunjie CAO,Junxiao LAI,Tianjiao YU. AntiGPS spoofing method for UAV based on LSTM-KF model [J]. Chinese Journal of Network and Information Security, 2020, 6(5): 80-88.
[7]	Wei HUANG,Cuncai LIU,Sibo QI. LSTM network traffic prediction and link congestion warning scheme for single port and single link [J]. Chinese Journal of Network and Information Security, 2019, 5(6): 50-57.