基于多级注意力融合机制的藏文实体关系抽取

doi:10.11959/j.issn.2096-6652.202146

智能科学与技术学报 ›› 2021, Vol. 3 ›› Issue (4): 466-473.doi: 10.11959/j.issn.2096-6652.202146

基于多级注意力融合机制的藏文实体关系抽取

王丽客¹^,², 孙媛¹^,², 刘思思¹^,²

¹ 中央民族大学信息工程学院，北京 100081
² 中央民族大学国家语言资源监测与研究少数民族语言中心，北京 100081

修回日期:2021-02-23 出版日期:2021-12-15 发布日期:2021-12-01
作者简介:王丽客（1996- ），女，中央民族大学信息工程学院硕士生，主要研究方向为自然语言处理、知识图谱
孙媛（1979- ），女，博士，中央民族大学信息工程学院副教授，主要研究方向为自然语言处理、知识图谱、问答系统
刘思思（1998- ），女，中央民族大学信息工程学院硕士生，主要研究方向为自然语言处理、知识图谱
基金资助:
国家自然科学基金资助项目(61972436)

Tibetan entity relation extraction based on multi-level attention fusion mechanism

Like WANG¹^,², Yuan SUN¹^,², Sisi LIU¹^,²

¹ School of Information Engineering, Minzu University of China, Beijing 100081, China
² National Language Resource and Monitoring and Research Center of Minority Languages, Minzu University of China, Beijing 100081, China

Revised:2021-02-23 Online:2021-12-15 Published:2021-12-01
Supported by:
The National Nature Science Foundation of China(61972436)

摘要/Abstract

摘要：

与中英文相比，藏文实体关系训练语料规模较小，传统有监督的学习方法难以获得较高的准确率。针对基于远程监督的实体关系抽取存在错误标记的问题，利用远程监督方法将知识库与文本对齐，构建藏文实体关系抽取的数据集，提出一个基于多级注意力融合机制的藏文实体关系抽取模型。在词级别引入自注意力机制来提取单词的内部特征，在句子级别引入注意力机制为每个实例分配权重，从而充分利用包含信息的句子，减少噪声实例的权重。同时引入联合评分函数，修正远程监督的错误标签，并将神经网络与支持向量机结合，实现藏文实体关系分类。实验结果表明，提出的模型有效提高了藏文实体关系抽取的准确率，且优于基线模型效果。

关键词: 藏文, 实体关系抽取, 多级注意力融合机制, 支持向量机

Abstract:

Compared with Chinese and English, the training corpus of Tibetan entity relation is smaller, so it is difficult to obtain higher accuracy based on traditional supervised learning methods.And there exists the problem of wrong labels in distant supervision for relation extraction.To solve these problems, the distant supervision method was used to construct the data set of Tibetan entity relation extraction through aligning the knowledge base with texts, which could alleviate the problem of lacking of large-scale corpus in Tibetan.And a Tibetan entity relation extraction model based on multi-level attention fusion mechanism was proposed.The self-attention was added to extract internal features of words in word level.The selective attention mechanism could assign weights of each instance, so as to make full use of informative sentences and reduce weights of noisy instances.Meanwhile, a joint score function was introduced to correct wrong labels, and neural network was combined with support vector machine to extract relations.Experimental results show that the proposed model can effectively improve the accuracy of Tibetan entity relation extraction, and is better than the baseline.

Key words: Tibetan, entity relation extraction, multi-level attention fusion mechanism, support vector machine

中图分类号:

王丽客,孙媛,刘思思. 基于多级注意力融合机制的藏文实体关系抽取[J]. 智能科学与技术学报, 2021, 3(4): 466-473.

Like WANG,Yuan SUN,Sisi LIU. Tibetan entity relation extraction based on multi-level attention fusion mechanism[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 466-473.

图/表 10

图1

图2

图3

表1

表2

表3

图4

表4

基于Bi-GRU的注意力机制模型与自注意力机制模型结果对比"

模型	F1	P@100	P@200	P@300	P@avg
PCNN+ATT	42.7%	48.6%	44%	41.7%	44.8%
+Bi-GRU+ATT	47.1%	54.4%	48.2%	46.8%	49.8%
+Self-ATT	$50 . 5 %$	$56 . 9 %$	$52 . 6 %$	$50 . 1 %$	$53 . 2 %$

表4

图5

表5

添加软标签方法和SVM分类器的实验结果"

模型	F1	P@100	P@200	P@300	P@avg
+soft-label	55.9%	59.8%	53.2%	49.9%	54.3%
+SVM	$62 . 6 %$	$64 . 1 %$	$59 . 7 %$	$53 . 8 %$	$59 . 2 %$

表5

参考文献 19

[1]	MINTZ M , BILLS S , SNOW R ,et al. Distant supervision for relation extraction without labeled data[C]// Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLPJ.[S.l.:s.n.], 2009: 1003-1011.
[2]	LIU C Y , SUN W B , CHAO W H ,et al. Convolution neural network for relation extraction[C]// Proceedings of the 9th International Conference on Advanced Data Mining and Applications,Part II.[S.l.:s.n.], 2013: 231-242.
[3]	ZENG D J , LIU K , LAI S W ,et al. Relation classification via convolutional deep neural network[C]// Proceedings of the 25th International Conference on Computational Linguistics:Technical Papers.[S.l.:s.n.], 2014: 2335-2344.
[4]	NGUYEN T H , GRISHMAN R . Relation extraction:perspective from convolutional neural networks[C]// Proceedings of NAACL-HLT.[S.l.:s.n.], 2015: 39-48.
[5]	ZHANG D X , WANG D . Relation classification via recurrent neural network[J]. arXiv preprint,2015,arXiv:1508.01006.
[6]	ZHOU P , SHI W , TIAN J ,et al. Attention-based bidirectional long short-term memory networks for relation classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2016: 207-212.
[7]	WANG L L , CAO Z , MELO G D ,et al. Relation classification via multi-level attention CNNs[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2016: 1298-1307.
[8]	ZHU J Z , QIAO J Z , DAI X X ,et al. Relation classification via target-concentrated attention CNNs[C]// Proceedings of the 31st International Conference on Neural Information Processing.[S.l.:s.n.], 2017: 137-146.
[9]	DIETTERICH T G , LATHROP R H , LOZANO-PEREZ T , . Solving the multiple instance problem with axis-parallel rectangles[J]. Journal of the Artificial Intelligence, 1997,89(1-2): 31-71.
[10]	RIEDEL S , YAO L M , MCCALLUM A . Modeling relations and their mentions without labeled text[C]// Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases.[S.l.:s.n.], 2010: 148-163.
[11]	ZENG D J , LIU K , CHEN Y B ,et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.[S.l.:s.n.], 2015: 1753-1762.
[12]	LIN Y K , SHEN S Q , LIU Z Y ,et al. Neural relation extraction with selective attention over instancess[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2016: 2124-2133.
[13]	JIANG X T , WANG Q , LI P ,et al. Relation Extraction with Multi-instance Multi-label Convolutional Neural Networks[C]// Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.[S.l.:s.n.], 2016: 1471-1480.
[14]	FENG X C , GUO J , QIN B ,et al. Effective deep memory networks for distant supervised relation extraction[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence.[S.l.:s.n.], 2017: 4002-4008.
[15]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems.[S.l.:s.n.], 2017: 5998-6008.
[16]	朱臻, 孙媛 . 基于 SVM 和泛化模板协作的藏语人物属性抽取[J]. 中文信息学报, 2015,29(6): 220-227.
	ZHU Z , SUN Y . Tibetan person attribute extraction based on SVM and pattern[J]. Journal of Chinese Information Processing, 2015,29(6): 220-227.
[17]	夏天赐, 孙媛 . 基于联合模型的藏文实体关系抽取方法研究[J]. 中文信息学报, 2018,32(12): 76-83.
	XIA T C , SUN Y . Tibetan entity relation extraction based on joint model[J]. Journal of Chinese Information Processing, 2018,32(12): 76-83.
[18]	郭莉莉, 孙媛 . 基于 BP 神经网络的藏语实体关系抽取[J]. 软件导刊, 2019,18(3): 7-9,15.
	GUO L L , SUN Y . Tibetan entity relation extraction based on BP neural network[J]. Software Guide, 2019,18(3): 7-9,15.
[19]	MATTHEW E P , NEUMANN M , IYYER M ,et al. Deep contextualized word representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.[S.l.:s.n.], 2018: 2227-2237.

参数名	参数值
Batchsize	50
Learning_rate	0.1
Dropout	0.5

参数名	参数值
Num_blocks	4
Num_heads	10
Hidden_size	230

模型	F1	P@100	P@200	P@300	P@avg
CNN+AVE	32.6%	47.5%	36.2%	32.5%	38.8%
CNN+ATT	36.2%	52.4%	39.8%	33.8%	42.0%
PCNN+AVE	37.9%	45.0%	42.5%	39.2%	42.2%
PCNN+ATT	42.7%	48.6%	44.0%	41.7%	44.8%

基于多级注意力融合机制的藏文实体关系抽取

Tibetan entity relation extraction based on multi-level attention fusion mechanism

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 19

相关文章 15

Metrics

推荐阅读 0

[1]	项凤涛, 苏炯铭, 谷学强, 张万鹏. 基于智能体建模的新冠肺炎疫情传播问题研究[J]. 智能科学与技术学报, 0, (): 51-57.
[2]	王飞跃, 缪青海, 张军平, 郑文博, 丁文文. 探讨AI for Science的影响与意义：现状与展望[J]. 智能科学与技术学报, 2023, 5(1): 1-6.
[3]	罗欣欢, 王奕璇, 李炜, 陈曦. 基于显著性信息的Fit CutMix数据增强算法在医学影像上的应用[J]. 智能科学与技术学报, 2023, 5(1): 58-68.
[4]	王晓, 杨林瑶, 胡斌, 侯家琛. 平行推理：一种基于ACP方法的虚实互动的知识协同框架[J]. 智能科学与技术学报, 2023, 5(1): 69-82.
[5]	顾宙瑜, 於跃成, 者甜甜. Rapider-YOLOX：高效的轻量级目标检测网络[J]. 智能科学与技术学报, 2023, 5(1): 92-103.
[6]	张军欢, 朱正一, 蔡可玮. 人工智能与量化交易的课程建设[J]. 智能科学与技术学报, 2023, 5(1): 104-112.
[7]	丛晓峰, 桂杰, 章军. 基于视觉Transformer的多损失融合水下图像增强网络[J]. 智能科学与技术学报, 2022, 4(4): 522-532.
[8]	于航, 付彦伟, 姜柏言, 薛向阳. 基于少量图像的三维重建综述[J]. 智能科学与技术学报, 2022, 4(4): 544-559.
[9]	张俊, 许沛东, 陈思远, 高天露, 戴宇欣, 张科, 赵杭, 高杰迈, 白昱阳, 李金星, 张浩然, 李湘, 陈玖香. 物理-数据-知识混合驱动的人机混合增强智能系统管控方法[J]. 智能科学与技术学报, 2022, 4(4): 571-583.
[10]	陈妍, 罗雪琴, 梁伟, 谢永芳. 基于情感信息融合注意力机制的抑郁症识别[J]. 智能科学与技术学报, 2022, 4(4): 600-609.
[11]	黄本遵, 陈德旺, 何振峰, 邓新国, GIUSEPPE CARLO Marano. 基于人机混合智能的地铁列车无人驾驶系统研究[J]. 智能科学与技术学报, 2022, 4(4): 584-591.
[12]	张永强, 宋美霖, 刘天虎, 满梦华. 混合骨骼特征的三帧间差分手势识别方法研究[J]. 智能科学与技术学报, 2022, 4(4): 592-599.
[13]	邵虹, 左常升, 张萍. 结合Attention U-Net与瓶颈检测的肺部细胞图像分割方法[J]. 智能科学与技术学报, 2022, 4(4): 610-616.
[14]	武强, 季雪庭, 吕琳媛. 元宇宙中的人工智能技术与应用[J]. 智能科学与技术学报, 2022, 4(3): 324-334.
[15]	赖文柱, 陈德旺, 何振峰, 邓新国, GIUSEPPE CARLO Marano. 地铁列车驾驶技术发展综述：从人工驾驶到智能无人驾驶[J]. 智能科学与技术学报, 2022, 4(3): 335-343.