融合混合嵌入与关系标签嵌入的三元组联合抽取方法

doi:10.11959/j.issn.1000-0801.2023021

电信科学 ›› 2023, Vol. 39 ›› Issue (2): 132-144.doi: 10.11959/j.issn.1000-0801.2023021

融合混合嵌入与关系标签嵌入的三元组联合抽取方法

戴剑锋, 陈星妤, 董黎刚, 蒋献

浙江工商大学，浙江杭州 310018

修回日期:2023-01-20 出版日期:2023-02-20 发布日期:2023-02-01
作者简介:戴剑锋（1997- ），男，浙江工商大学信息与电子工程学院（萨塞克斯人工智能学院）硕士生，主要研究方向为智慧教育、自然语言处理
陈星妤（1999- ），女，浙江工商大学信息与电子工程学院（萨塞克斯人工智能学院）硕士生，主要研究方向为智慧教育、自然语言处理
董黎刚（1973- ），男，博士，浙江工商大学信息与电子工程学院（萨塞克斯人工智能学院）党委书记、教授、博士生导师，浙江省计算机学会理事，主要研究方向为新一代网络和分布式系统
蒋献（1988- ），男，浙江工商大学信息与电子工程学院（萨塞克斯人工智能学院）讲师、实验员，主要研究方向为智慧教育和智慧网络
基金资助:
国家社会科学基金资助项目(17BYY090);浙江省重点研发计划项目(2017C03058);浙江省“尖兵”“领雁”研发攻关计划项目(2023C03202)

A triple joint extraction method combining hybrid embedding and relational label embedding

Jianfeng DAI, Xingyu CHEN, Ligang DONG, Xian JIANG

Zhejiang Gongshang University, Hangzhou 310018, China

Revised:2023-01-20 Online:2023-02-20 Published:2023-02-01
Supported by:
The National Social Science Foundation of China(17BYY090);Zhejiang Province Key Research and Development Program(2017C03058);Zhejiang Province “Top Soldiers” and “Leading Geese” Project(2023C03202)

摘要/Abstract

摘要：

三元组抽取的目的是从非结构化的文本中获取实体与实体间的关系，并应用于下游任务。嵌入机制对三元组抽取模型的性能有很大影响，嵌入向量应包含与关系抽取任务密切相关的丰富语义信息。在中文数据集中，字词之间包含的信息有很大区别，为了改进由分词错误产生的语义信息丢失问题，设计了融合混合嵌入与关系标签嵌入的三元组联合抽取方法（HEPA），提出了采用字嵌入与词嵌入结合的混合嵌入方法，降低由分词错误产生的误差；在实体抽取层中添加关系标签嵌入机制，融合文本与关系标签，利用注意力机制来区分句子中实体与不同关系标签的相关性，由此提高匹配精度；采用指针标注的方法匹配实体，提高了对关系重叠三元组的抽取效果。在公开的DuIE数据集上进行了对比实验，相较于表现最好的基线模型（CasRel）， HEPA的F1值提升了2.8%。

关键词: 三元组抽取, 关系嵌入, BERT, 注意力机制, 指针标注

Abstract:

The purpose of triple extraction is to obtain relationships between entities from unstructured text and apply them to downstream tasks.The embedding mechanism has a great impact on the performance of the triple extraction model, and the embedding vector should contain rich semantic information that is closely related to the relationship extraction task.In Chinese datasets, the information contained between words is very different, and in order to avoid the loss of semantic information problems generated by word separation errors, a triple joint extraction method combining hybrid embedding and relational label embedding (HEPA) was designed, and a hybrid embedding means that combines letter embedding and word embedding was proposed to reduce the errors generated by word separation errors.A relational embedding mechanism that fuses text and relational labels was added, and an attention mechanism was used to distinguish the relevance of entities in a sentence with different relational labels, thus improving the matching accuracy.The method of matching entities with pointer annotation was used, which improved the extraction effect on relational overlapping triples.Comparative experiments are conducted on the publicly available DuIE dataset, and the F1 value of HEPA is improved by 2.8% compared to the best performing baseline model (CasRel).

Key words: triple extraction, relational embedding, BERT, attention mechanism, pointer annotation

中图分类号:

TP393

戴剑锋, 陈星妤, 董黎刚, 蒋献. 融合混合嵌入与关系标签嵌入的三元组联合抽取方法[J]. 电信科学, 2023, 39(2): 132-144.

Jianfeng DAI, Xingyu CHEN, Ligang DONG, Xian JIANG. A triple joint extraction method combining hybrid embedding and relational label embedding[J]. Telecommunications Science, 2023, 39(2): 132-144.

图/表 7

图1

图2

图3

表1

表2

表3

图4

参考文献 27

[1]	李冬梅, 张扬, 李东远 ,等. 实体关系抽取方法研究综述[J]. 计算机研究与发展, 2020,57(7): 1424-1448.
	LI D M , ZHANG Y , LI D Y ,et al. Overview of entity relationship extraction methods[J]. Computer Research and Development, 2020,57(7): 1424-1448.
[2]	ZENG D J , LIU K , LAI S W ,et al. Relation classification via convolutional deep neural network[C]// Proceedings of International Conference on Computational Linguistics.[S.l.:s.n.], 2014.
[3]	XU K , FENG Y , HUANG S ,et al. Semantic relation classification via convolutional neural networks with simple negative sampling[J]. Computer Science, 2015,71(7): 941-9.
[4]	SOCHER R , HUVAL B , MANNING C D ,et al. Semantic compositionality through recursive matrix-vector spaces[C]// Proceedings of Joint Conference on Empirical Methods in Natural Language Processing ＆ Computational Natural Language Learning. Hongkong:EMNLP Press, 2012.
[5]	LI F , ZHANG M , FU G ,et al. A Bi-LSTM-RNN model for relation classification using low-cost sequence features:1048550/arXiv160807720[P]. 2016.
[6]	SU Z , JIANG J . Hierarchical gated recu0rrent unit with semantic attention for event prediction[J]. Future Internet, 2020,12(2): 39.
[7]	VASHISHTH S , JOSHI R , PRAYAGA S S ,et al. RESIDE:improving distantly-supervised neural relation extraction using side information[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.[S.l.:s.n.], 2018.
[8]	杨帅, 王瑞琴, 马辉 . 基于多通道的边学习图卷积网络[J]. 电信科学, 2022,38(9): 95-104.
	YANG S , WANG R Q , MA H . Multi-channel based edge-learning graph convolutional network[J]. Telecommunications Science, 2022,38(9): 95-104.
[9]	李昊, 陈艳平, 唐瑞雪 ,等. 基于实体边界组合的关系抽取方法[J]. 计算机应用, 2022,42(6): 6.
	LI H , CHEN Y P , TANG R X ,et al. Relationship extraction method based on entity boundary combination[J]. Computer Applications, 2022,42(6): 6.
[10]	ZHONG Z , CHEN D . A frustratingly easy approach for entity and relation extraction[C]// Proceedings of the North American Chapter of the Association for Computational Linguistics.[S.l.:s.n.], 2021.
[11]	MIWA M , BANSAL M . End-to-end relation extraction using LSTMs on sequences and tree structures[J]. arXiv preprint,arXiv:1601.00770, 2016.
[12]	KATIYAR A , CARDIE C . Going out on a limb:joint extraction of entity mentions and relations without dependency trees[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).[S.l.:s.n.], 2017: 917-928.
[13]	ZHENG S , F WANG , BAO H ,et al. Joint extraction of entities and relations based on a novel tagging scheme[J]. arXiv preprint,arXiv:1706.05075, 2017.
[14]	ZENG X , ZENG D , HE S ,et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).[S.l.:s.n.], 2018: 506-514.
[15]	FU T J , MA W Y . GraphRel:modeling text as relational graphs for joint entity and relation extraction[C]// Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2019: 1409-1418.
[16]	DUAN G , MIAO J , HUANG T ,et al. A relational adaptive neural model for joint entity and relation extraction[J]. Frontiers in Neurorobotics, 2021(15): 635492.
[17]	WEI Z , SU J , WANG Y ,et al. A novel cascade binary tagging framework for relational triple extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2020: 1476-1488.
[18]	WANG Y , YU B , ZHANG Y ,et al. TPLinker:single-stage joint extraction of entities and relations through token pair linking[J]. arXiv preprint,arXiv:2010.13415, 2020.
[19]	田佳来, 吕学强, 游新冬 ,等. 基于分层序列标注的实体关系联合抽取方法[J]. 北京大学学报：自然科学版, 2021,57(1): 53-60.
	TIAN J L , LYU X Q , YOU X D ,et al. A joint extraction method of entity relations based on hierarchical sequence annotation[J]. Journal of Peking University (Natural Science Edition), 2021,57(1): 53-60.
[20]	苗琳, 张英俊, 谢斌红 ,等. 基于图神经网络的联合实体关系抽取[J]. 计算机应用研究, 2022,39(2): 424-431.
	MIAO L , ZHANG Y J , XIE B H ,et al. Joint entity relationship extraction based on graph neural network[J]. Proceedings of the Computer Application Research, 2022,39(2): 424-431.
[21]	王红, 吴燕婷 . 基于多跳注意力的实体关系联合抽取方法及应用研究[J]. 太原理工大学学报, 2022,53(1): 63-70.
	WANG H , WU Y T . Joint extraction of entity relationships based on multi-hop attention and its application[J]. Proceedings of the Journal of Taiyuan University of Technology, 2022,53(1): 63-70.
[22]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint,arXiv:1810.04805, 2018.
[23]	ZENG D , ZHANG H , LIU Q . CopyMTL:copy mechanism for joint extraction of entities and relations with multi-task learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(5): 9507-9514.
[24]	NAYAK T , NG H T . Effective modeling of encoder-decoder architecture for joint entity and relation extraction[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(5): 8528-8535.
[25]	REN X , WU Z , HE W ,et al. CoType:joint extraction of typed entities and relations with knowledge bases[J]. Proceedings of the 26th International Conference on World Wide Web.New York:ACM Press, 2017: 1015-1024.
[26]	GIANNIS B , JOHANNES D , THOMAS D ,et al. Joint entity recognition and relation extraction as a multi-head selection problem[J]. Expert Systems with Application, 2018,114(11): 34-45.
[27]	WEI Z , SU J , WANG Y ,et al. A novel hierarchical binary tagging framework for relational triple extraction[J]. arXiv preprint,arXiv:1909.03227v4, 2020.

模型	精确率	召回率	F1值
CopyMTL^[23]	0.496	0.394	0.439
WDec^[24]	0.641	0.542	0.587
CoType^[25]	0.659	0.609	0.633
MHS^[26]	0.772	0.623	0.690
CasRel^[27]	0.786	0.689	0.734
HEPA	0.793	0.733	0.762

模型	(s,r)	(r,o)	(s,o)	(s,r,o)
CopyMTL	0.482	0.489	0.502	0.439
WDec	0.623	0.633	0.652	0.587
CoType	0.673	0.669	0.675	0.633
MHS	0.725	0.737	0.742	0.690
CasRel	0.761	0.782	0.789	0.734
HEPA（无关系嵌入）	0.731	0.722	0.735	0.709
HEPA	0.792	0.803	0.809	0.762

模型	精确率	召回率	F1值
去除词嵌入	0.742	0.682	0.711
去除字嵌入	0.763	0.692	0.725
去除实体位置注意力机制	0.775	0.659	0.704
去除关系嵌入机制	0.732	0.694	0.709
HEPA	0.793	0.733	0.762

融合混合嵌入与关系标签嵌入的三元组联合抽取方法

A triple joint extraction method combining hybrid embedding and relational label embedding

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 27

相关文章 7

Metrics

推荐阅读 0

[1]	张永, 刘纪奎, 柯文龙. 基于并行可分离卷积和标签平滑正则化的脑电情感识别[J]. 电信科学, 2023, 39(5): 116-128.
[2]	宋勇, 严志伟, 秦玉坤, 赵东明, 叶晓舟, 柴园园, 欧阳晔. 基于矩阵分解和注意力多任务学习的客服投诉工单分类[J]. 电信科学, 2022, 38(2): 103-110.
[3]	金楠, 王瑞琴, 陆悦聪. 基于艾宾浩斯遗忘曲线和注意力机制的推荐算法[J]. 电信科学, 2022, 38(10): 89-97.
[4]	孙金杨, 刘柏嵩, 任豪, 钱江波. TAGAN：一种融合细粒度语义特征的学术论文对抗推荐算法[J]. 电信科学, 2021, 37(8): 57-65.
[5]	蔡鑫. 基于Bert模型的互联网不良信息检测[J]. 电信科学, 2020, 36(11): 121-126.
[6]	王一鸣,陈恳,萨阿卜杜萨拉木·艾海提拉木. 基于SDBN和BLSTM注意力融合的端到端视听双模态语音识别[J]. 电信科学, 2019, 35(12): 79-89.
[7]	杨汉华,闫述. 矿用RFID标签天线的分析与设计[J]. 电信科学, 2009, 25(1): 70-73.