基于多级注意力融合机制的藏文实体关系抽取

doi:10.11959/j.issn.2096-6652.202146

Abstract

Abstract:

Compared with Chinese and English, the training corpus of Tibetan entity relation is smaller, so it is difficult to obtain higher accuracy based on traditional supervised learning methods.And there exists the problem of wrong labels in distant supervision for relation extraction.To solve these problems, the distant supervision method was used to construct the data set of Tibetan entity relation extraction through aligning the knowledge base with texts, which could alleviate the problem of lacking of large-scale corpus in Tibetan.And a Tibetan entity relation extraction model based on multi-level attention fusion mechanism was proposed.The self-attention was added to extract internal features of words in word level.The selective attention mechanism could assign weights of each instance, so as to make full use of informative sentences and reduce weights of noisy instances.Meanwhile, a joint score function was introduced to correct wrong labels, and neural network was combined with support vector machine to extract relations.Experimental results show that the proposed model can effectively improve the accuracy of Tibetan entity relation extraction, and is better than the baseline.

Key words: Tibetan, entity relation extraction, multi-level attention fusion mechanism, support vector machine

CLC Number:

Like WANG,Yuan SUN,Sisi LIU. Tibetan entity relation extraction based on multi-level attention fusion mechanism[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 466-473.

Figures/Tables 10

模型	F1	P@100	P@200	P@300	P@avg
PCNN+ATT	42.7%	48.6%	44%	41.7%	44.8%
+Bi-GRU+ATT	47.1%	54.4%	48.2%	46.8%	49.8%
+Self-ATT	$50 . 5 %$	$56 . 9 %$	$52 . 6 %$	$50 . 1 %$	$53 . 2 %$

模型	F1	P@100	P@200	P@300	P@avg
+soft-label	55.9%	59.8%	53.2%	49.9%	54.3%
+SVM	$62 . 6 %$	$64 . 1 %$	$59 . 7 %$	$53 . 8 %$	$59 . 2 %$

References 19

[1]	MINTZ M , BILLS S , SNOW R ,et al. Distant supervision for relation extraction without labeled data[C]// Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLPJ.[S.l.:s.n.], 2009: 1003-1011.
[2]	LIU C Y , SUN W B , CHAO W H ,et al. Convolution neural network for relation extraction[C]// Proceedings of the 9th International Conference on Advanced Data Mining and Applications,Part II.[S.l.:s.n.], 2013: 231-242.
[3]	ZENG D J , LIU K , LAI S W ,et al. Relation classification via convolutional deep neural network[C]// Proceedings of the 25th International Conference on Computational Linguistics:Technical Papers.[S.l.:s.n.], 2014: 2335-2344.
[4]	NGUYEN T H , GRISHMAN R . Relation extraction:perspective from convolutional neural networks[C]// Proceedings of NAACL-HLT.[S.l.:s.n.], 2015: 39-48.
[5]	ZHANG D X , WANG D . Relation classification via recurrent neural network[J]. arXiv preprint,2015,arXiv:1508.01006.
[6]	ZHOU P , SHI W , TIAN J ,et al. Attention-based bidirectional long short-term memory networks for relation classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2016: 207-212.
[7]	WANG L L , CAO Z , MELO G D ,et al. Relation classification via multi-level attention CNNs[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2016: 1298-1307.
[8]	ZHU J Z , QIAO J Z , DAI X X ,et al. Relation classification via target-concentrated attention CNNs[C]// Proceedings of the 31st International Conference on Neural Information Processing.[S.l.:s.n.], 2017: 137-146.
[9]	DIETTERICH T G , LATHROP R H , LOZANO-PEREZ T , . Solving the multiple instance problem with axis-parallel rectangles[J]. Journal of the Artificial Intelligence, 1997,89(1-2): 31-71.
[10]	RIEDEL S , YAO L M , MCCALLUM A . Modeling relations and their mentions without labeled text[C]// Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases.[S.l.:s.n.], 2010: 148-163.
[11]	ZENG D J , LIU K , CHEN Y B ,et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.[S.l.:s.n.], 2015: 1753-1762.
[12]	LIN Y K , SHEN S Q , LIU Z Y ,et al. Neural relation extraction with selective attention over instancess[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.:s.n.], 2016: 2124-2133.
[13]	JIANG X T , WANG Q , LI P ,et al. Relation Extraction with Multi-instance Multi-label Convolutional Neural Networks[C]// Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.[S.l.:s.n.], 2016: 1471-1480.
[14]	FENG X C , GUO J , QIN B ,et al. Effective deep memory networks for distant supervised relation extraction[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence.[S.l.:s.n.], 2017: 4002-4008.
[15]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems.[S.l.:s.n.], 2017: 5998-6008.
[16]	朱臻, 孙媛 . 基于 SVM 和泛化模板协作的藏语人物属性抽取[J]. 中文信息学报, 2015,29(6): 220-227.
	ZHU Z , SUN Y . Tibetan person attribute extraction based on SVM and pattern[J]. Journal of Chinese Information Processing, 2015,29(6): 220-227.
[17]	夏天赐, 孙媛 . 基于联合模型的藏文实体关系抽取方法研究[J]. 中文信息学报, 2018,32(12): 76-83.
	XIA T C , SUN Y . Tibetan entity relation extraction based on joint model[J]. Journal of Chinese Information Processing, 2018,32(12): 76-83.
[18]	郭莉莉, 孙媛 . 基于 BP 神经网络的藏语实体关系抽取[J]. 软件导刊, 2019,18(3): 7-9,15.
	GUO L L , SUN Y . Tibetan entity relation extraction based on BP neural network[J]. Software Guide, 2019,18(3): 7-9,15.
[19]	MATTHEW E P , NEUMANN M , IYYER M ,et al. Deep contextualized word representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.[S.l.:s.n.], 2018: 2227-2237.

Metrics

Recommended 0

No Suggested Reading articles found!

参数名	参数值
Batchsize	50
Learning_rate	0.1
Dropout	0.5

参数名	参数值
Num_blocks	4
Num_heads	10
Hidden_size	230

模型	F1	P@100	P@200	P@300	P@avg
CNN+AVE	32.6%	47.5%	36.2%	32.5%	38.8%
CNN+ATT	36.2%	52.4%	39.8%	33.8%	42.0%
PCNN+AVE	37.9%	45.0%	42.5%	39.2%	42.2%
PCNN+ATT	42.7%	48.6%	44.0%	41.7%	44.8%

Tibetan entity relation extraction based on multi-level attention fusion mechanism

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 19

Related Articles 15

Metrics

Recommended 0

[1]	Qiang WU, Xueting JI, Linyuan LYU. Artificial intelligence technologies and applications in the metaverse [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 324-334.
[2]	Wenzhu LAI, Dewang CHEN, Zhenfeng HE, Xinguo DENG, CARLO Marano GIUSEPPE. Overview of metro train driving technology development:from manual driving to intelligent unmanned driving [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 335-343.
[3]	Chao GUO, Yue LU, Xiao WANG, Da YI, Xiao WANG, Fei-Yue WANG. Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 344-354.
[4]	Qiuyun LYU, Shaopeng CHENG, Manzhi YANG, Xiaoguang CHEN, Zhen WANG. A metaverse-oriented digital citizen authentication scheme [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 396-409.
[5]	Zhou YU, Jing BI, Haitao YUAN. A path planning method for complex naval battle field based on an improved DQN algorithm [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 418-425.
[6]	Xin ZHANG, Zhihui ZHAN. Application of intelligent optimization algorithms in supply chain network [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 174-183.
[7]	Renwu LI, Lingxiao ZHANG, Lin GAO, Chunpeng LI, Hao JIANG. Category-level object pose estimation from depth point cloud [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 246-254.
[8]	Yuanwen CHEN, Xiao WANG, Lingxi LI, Fei-Yue WANG. Traffic situational awareness research and development enhanced by social media data: the state of the art and prospects [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 1-13.
[9]	Qiang YANG, Yongxin TONG, Yansheng WANG, Lixin FAN, Wei WANG, Lei CHEN, Wei WANG, Yan KANG. A survey on federated learning in crowd intelligence [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 29-44.
[10]	Mingyang CHEN, Wen ZHANG, Xiangnan CHEN, Hongting ZHOU, Huajun CHEN. Collective knowledge graph: meta knowledge transfer and federated graph reasoning [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 55-64.
[11]	Yue YU, Xin LIU, Fangqing JIANG, Han ZHANG, Hui WANG, Wei ZENG. Exploration of the continual learning ability that supports the application ecological evolution of the large-scale pretraining Peng Cheng series open source models [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 97-108.
[12]	Hang ZHAO, Sheng LIU, Kun LUO, Shichao CHEN, Linghui KONG, Fan JIA. Research on application of edge computing system based on KubeEdge [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 118-128.
[13]	Xiaoshuang LI, Xiao WANG, Linyao YANG, Yonglin TIAN, Yutong WANG, Jun ZHANG, Fei-Yue WANG. MetaGrid: a parallel grids based approach for next generation smart power systems [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 387-398.
[14]	Bowen XU, Jing BI, Haitao YUAN, Gongming WANG, Junfei QIAO. Dynamic water quality warning with seasonal decomposition and long short-term memory network [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 456-465.
[15]	Hong SHAO, Mingkun ZHANG, Wencheng CUI. Classification method of dermoscopic image based on hierarchical convolution neural network [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 474-481.