融合混合嵌入与关系标签嵌入的三元组联合抽取方法

doi:10.11959/j.issn.1000-0801.2023021

Abstract

Abstract:

The purpose of triple extraction is to obtain relationships between entities from unstructured text and apply them to downstream tasks.The embedding mechanism has a great impact on the performance of the triple extraction model, and the embedding vector should contain rich semantic information that is closely related to the relationship extraction task.In Chinese datasets, the information contained between words is very different, and in order to avoid the loss of semantic information problems generated by word separation errors, a triple joint extraction method combining hybrid embedding and relational label embedding (HEPA) was designed, and a hybrid embedding means that combines letter embedding and word embedding was proposed to reduce the errors generated by word separation errors.A relational embedding mechanism that fuses text and relational labels was added, and an attention mechanism was used to distinguish the relevance of entities in a sentence with different relational labels, thus improving the matching accuracy.The method of matching entities with pointer annotation was used, which improved the extraction effect on relational overlapping triples.Comparative experiments are conducted on the publicly available DuIE dataset, and the F1 value of HEPA is improved by 2.8% compared to the best performing baseline model (CasRel).

Key words: triple extraction, relational embedding, BERT, attention mechanism, pointer annotation

CLC Number:

TP393

Jianfeng DAI, Xingyu CHEN, Ligang DONG, Xian JIANG. A triple joint extraction method combining hybrid embedding and relational label embedding[J]. Telecommunications Science, 2023, 39(2): 132-144.

Figures/Tables 7

References 27

[1]	李冬梅, 张扬, 李东远 ,等. 实体关系抽取方法研究综述[J]. 计算机研究与发展, 2020,57(7): 1424-1448.
	LI D M , ZHANG Y , LI D Y ,et al. Overview of entity relationship extraction methods[J]. Computer Research and Development, 2020,57(7): 1424-1448.
[2]	ZENG D J , LIU K , LAI S W ,et al. Relation classification via convolutional deep neural network[C]// Proceedings of International Conference on Computational Linguistics.[S.l.:s.n.], 2014.
[3]	XU K , FENG Y , HUANG S ,et al. Semantic relation classification via convolutional neural networks with simple negative sampling[J]. Computer Science, 2015,71(7): 941-9.
[4]	SOCHER R , HUVAL B , MANNING C D ,et al. Semantic compositionality through recursive matrix-vector spaces[C]// Proceedings of Joint Conference on Empirical Methods in Natural Language Processing ＆ Computational Natural Language Learning. Hongkong:EMNLP Press, 2012.
[5]	LI F , ZHANG M , FU G ,et al. A Bi-LSTM-RNN model for relation classification using low-cost sequence features:1048550/arXiv160807720[P]. 2016.
[6]	SU Z , JIANG J . Hierarchical gated recu0rrent unit with semantic attention for event prediction[J]. Future Internet, 2020,12(2): 39.
[7]	VASHISHTH S , JOSHI R , PRAYAGA S S ,et al. RESIDE:improving distantly-supervised neural relation extraction using side information[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.[S.l.:s.n.], 2018.
[8]	杨帅, 王瑞琴, 马辉 . 基于多通道的边学习图卷积网络[J]. 电信科学, 2022,38(9): 95-104.
	YANG S , WANG R Q , MA H . Multi-channel based edge-learning graph convolutional network[J]. Telecommunications Science, 2022,38(9): 95-104.
[9]	李昊, 陈艳平, 唐瑞雪 ,等. 基于实体边界组合的关系抽取方法[J]. 计算机应用, 2022,42(6): 6.
	LI H , CHEN Y P , TANG R X ,et al. Relationship extraction method based on entity boundary combination[J]. Computer Applications, 2022,42(6): 6.
[10]	ZHONG Z , CHEN D . A frustratingly easy approach for entity and relation extraction[C]// Proceedings of the North American Chapter of the Association for Computational Linguistics.[S.l.:s.n.], 2021.
[11]	MIWA M , BANSAL M . End-to-end relation extraction using LSTMs on sequences and tree structures[J]. arXiv preprint,arXiv:1601.00770, 2016.
[12]	KATIYAR A , CARDIE C . Going out on a limb:joint extraction of entity mentions and relations without dependency trees[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).[S.l.:s.n.], 2017: 917-928.
[13]	ZHENG S , F WANG , BAO H ,et al. Joint extraction of entities and relations based on a novel tagging scheme[J]. arXiv preprint,arXiv:1706.05075, 2017.
[14]	ZENG X , ZENG D , HE S ,et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).[S.l.:s.n.], 2018: 506-514.
[15]	FU T J , MA W Y . GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]// Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2019: 1409-1418.
[16]	DUAN G , MIAO J , HUANG T ,et al. A relational adaptive neural model for joint entity and relation extraction[J]. Frontiers in Neurorobotics, 2021(15): 635492.
[17]	WEI Z , SU J , WANG Y ,et al. A novel cascade binary tagging framework for relational triple extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2020: 1476-1488.
[18]	WANG Y , YU B , ZHANG Y ,et al. TPLinker:single-stage joint extraction of entities and relations through token pair linking[J]. arXiv preprint,arXiv:2010.13415, 2020.
[19]	田佳来, 吕学强, 游新冬 ,等. 基于分层序列标注的实体关系联合抽取方法[J]. 北京大学学报：自然科学版, 2021,57(1): 53-60.
	TIAN J L , LYU X Q , YOU X D ,et al. A joint extraction method of entity relations based on hierarchical sequence annotation[J]. Journal of Peking University (Natural Science Edition), 2021,57(1): 53-60.
[20]	苗琳, 张英俊, 谢斌红 ,等. 基于图神经网络的联合实体关系抽取[J]. 计算机应用研究, 2022,39(2): 424-431.
	MIAO L , ZHANG Y J , XIE B H ,et al. Joint entity relationship extraction based on graph neural network[J]. Proceedings of the Computer Application Research, 2022,39(2): 424-431.
[21]	王红, 吴燕婷 . 基于多跳注意力的实体关系联合抽取方法及应用研究[J]. 太原理工大学学报, 2022,53(1): 63-70.
	WANG H , WU Y T . Joint extraction of entity relationships based on multi-hop attention and its application[J]. Proceedings of the Journal of Taiyuan University of Technology, 2022,53(1): 63-70.
[22]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint,arXiv:1810.04805, 2018.
[23]	ZENG D , ZHANG H , LIU Q . CopyMTL:copy mechanism for joint extraction of entities and relations with multi-task learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(5): 9507-9514.
[24]	NAYAK T , NG H T . Effective modeling of encoder-decoder architecture for joint entity and relation extraction[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(5): 8528-8535.
[25]	REN X , WU Z , HE W ,et al. CoType:joint extraction of typed entities and relations with knowledge bases[J]. Proceedings of the 26th International Conference on World Wide Web.New York:ACM Press, 2017: 1015-1024.
[26]	GIANNIS B , JOHANNES D , THOMAS D ,et al. Joint entity recognition and relation extraction as a multi-head selection problem[J]. Expert Systems with Application, 2018,114(11): 34-45.
[27]	WEI Z , SU J , WANG Y ,et al. A novel hierarchical binary tagging framework for relational triple extraction[J]. arXiv preprint,arXiv:1909.03227v4, 2020.

Metrics

Recommended 0

No Suggested Reading articles found!

模型	精确率	召回率	F1值
CopyMTL^[23]	0.496	0.394	0.439
WDec^[24]	0.641	0.542	0.587
CoType^[25]	0.659	0.609	0.633
MHS^[26]	0.772	0.623	0.690
CasRel^[27]	0.786	0.689	0.734
HEPA	0.793	0.733	0.762

模型	(s,r)	(r,o)	(s,o)	(s,r,o)
CopyMTL	0.482	0.489	0.502	0.439
WDec	0.623	0.633	0.652	0.587
CoType	0.673	0.669	0.675	0.633
MHS	0.725	0.737	0.742	0.690
CasRel	0.761	0.782	0.789	0.734
HEPA（无关系嵌入）	0.731	0.722	0.735	0.709
HEPA	0.792	0.803	0.809	0.762

模型	精确率	召回率	F1值
去除词嵌入	0.742	0.682	0.711
去除字嵌入	0.763	0.692	0.725
去除实体位置注意力机制	0.775	0.659	0.704
去除关系嵌入机制	0.732	0.694	0.709
HEPA	0.793	0.733	0.762

A triple joint extraction method combining hybrid embedding and relational label embedding

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 27

Related Articles 8

Metrics

Recommended 0

[1]	Yong ZHANG, Jikui LIU, Wenlong KE. EEG emotion recognition based on parallel separable convolution and label smoothing regularization [J]. Telecommunications Science, 2023, 39(5): 116-128.
[2]	Yong SONG, Zhiwei YAN, Yukun QIN, Dongming ZHAO, Xiaozhou YE, Yuanyuan CHAI, Ye OUYANG. Customer service complaint work order classification based on matrix factorization and attention multi-task learning [J]. Telecommunications Science, 2022, 38(2): 103-110.
[3]	Nan JIN, Ruiqin WANG, Yuecong LU. Ebbinghaus forgetting curve and attention mechanism based recommendation algorithm [J]. Telecommunications Science, 2022, 38(10): 89-97.
[4]	Jinyang SUN, Baisong LIU, Hao REN, Jiangbo QIAN. TAGAN: an academic paper adversarial recommendation algorithm incorporating fine-grained semantic features [J]. Telecommunications Science, 2021, 37(8): 57-65.
[5]	Xin CAI. Internet bad information detection based on Bert model [J]. Telecommunications Science, 2020, 36(11): 121-126.
[6]	Jinhua WANG,Na YING,Chendu ZHU,Zhaosen LIU,Zhedong CAI. Speech emotion recognition algorithm based on spectrogram feature extraction of deep space attention feature [J]. Telecommunications Science, 2019, 35(7): 100-108.
[7]	Yiming WANG,Ken CHEN,Aihaiti ABUDUSALAMU. End-to-end audiovisual speech recognition based on attention fusion of SDBN and BLSTM [J]. Telecommunications Science, 2019, 35(12): 79-89.
[8]	Hanhua Yang,Shu Yan. Analysis and Design Of RFID Tag Antenna fOr Mine MOnitOring Systems [J]. Telecommunications Science, 2009, 25(1): 70-73.