基于层级注意力机制的链接预测模型研究

doi:10.11959/j.issn.1000-436x.2021057

通信学报 ›› 2021, Vol. 42 ›› Issue (3): 36-44.doi: 10.11959/j.issn.1000-436x.2021057

基于层级注意力机制的链接预测模型研究

赵晓娟¹^,², 贾焰¹, 李爱平¹, 陈恺¹

¹ 国防科技大学计算机学院，湖南长沙 410073
² 湖南工业大学商学院，湖南株洲 412007

修回日期:2020-11-06 出版日期:2021-03-25 发布日期:2021-03-01
作者简介:赵晓娟（1975- ），女，湖南娄底人，国防科技大学博士生，主要研究方向为知识图谱表示、知识图谱推理、网络空间安全。
贾焰（1960- ），女，四川成都人，博士，国防科技大学教授、博士生导师，主要研究方向为数据挖掘、大数据分析、信息安全等。
李爱平（1974- ），男，山东诸城人，博士，国防科技大学研究员，主要研究方向为大数据分析、数据挖掘和网络信息安全。
陈恺（1994- ），男，湖北十堰人，国防科技大学硕士生，主要研究方向为知识图谱推理、网络空间安全。
基金资助:
国家重点研发计划基金资助项目(2017YFB0802204);国家重点研发计划基金资助项目(2016QY03D0603);国家重点研发计划基金资助项目(2016QY03D0601);国家重点研发计划基金资助项目(2017YFB0803301);国家重点研发计划基金资助项目(2019QY1406);广东省重点研发计划基金资助项目(2019B010136003);国家自然科学基金资助项目(61732004);国家自然科学基金资助项目(61732022);国家自然科学基金资助项目(61672020);湖南省重点研发计划基金资助项目(2018GK2056);湖南省教育厅科研基金资助项目(19C0597)

Research on link prediction model based on hierarchical attention mechanism

Xiaojuan ZHAO¹^,², Yan JIA¹, Aiping LI¹, Kai CHEN¹

¹ College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China
² College of Business, Hunan University of Technology, Zhuzhou 412007, China

Revised:2020-11-06 Online:2021-03-25 Published:2021-03-01
Supported by:
The National Key Research and Development Program of China(2017YFB0802204);The National Key Research and Development Program of China(2016QY03D0603);The National Key Research and Development Program of China(2016QY03D0601);The National Key Research and Development Program of China(2017YFB0803301);The National Key Research and Development Program of China(2019QY1406);The Key Research and Development Program of Guangdong Province(2019B010136003);The National Natural Science Foundation of China(61732004);The National Natural Science Foundation of China(61732022);The National Natural Science Foundation of China(61672020);The Key Research and Development Project of Hunan Province(2018GK2056);Scientific Research Found of Hunan Provincial Education Department(19C0597)

摘要/Abstract

摘要：

为了解决已有图注意力机制在进行链接预测相关任务时，容易造成注意力分配向某些出现频率高的关系倾斜的问题，提出了一种基于层级注意力机制的链接预测模型。在链接预测任务中，通过设计分层注意力机制，根据预测任务中的关系对知识图谱中与给定实体相连的不同类型的关系给予不同的注意力。在关注多跳邻居实体特征的同时，更关注关系特征以找到符合目标关系的关系类型。在多个基准数据集上与主流模型进行对比实验，实验结果表明，所提模型性能优于主流模型，并具有较好的稳健性。

关键词: 层级注意力机制, 链接预测, 知识图谱嵌入

Abstract:

In order to solve the problem that the existing graph attention mechanism tends to cause attention distribution to certain relations with high frequency when performing link prediction related tasks, a new link prediction model based on hierarchical attention mechanism was proposed.In the link prediction task, a hierarchical attention mechanism was designed to give different attention to the relationships of different relationship types connected to a given entity in the knowledge graph according to the relationship in the prediction task.While the characteristics of multi-hop neighbor entities were pay attention to, the relationship characteristics was pay more attention to find the relationship type that matches the target relationship.Through comparison experiments with the mainstream models on multiple benchmark data sets, the results show that the performance of the model is better than the mainstream models and has good robustness.

Key words: hierarchical attention mechanism, link prediction, knowledge graph embedding

中图分类号:

TP391

赵晓娟, 贾焰, 李爱平, 陈恺. 基于层级注意力机制的链接预测模型研究[J]. 通信学报, 2021, 42(3): 36-44.

Xiaojuan ZHAO, Yan JIA, Aiping LI, Kai CHEN. Research on link prediction model based on hierarchical attention mechanism[J]. Journal on Communications, 2021, 42(3): 36-44.

图/表 9

图1

图2

图3

表1

表2

表3

表4

表5

图4

参考文献 31

[1]	BAO J , DUAN N , YAN Z ,et al. Constraint-based question answering with knowledge graph[C]// International Conference on Computational Linguistics. Saarland:DBLP, 2016: 2503-2514.
[2]	WEST R , GABRILOVICH E , MURPHY K ,et al. Knowledge base completion via search-based question answering[C]// The 23rd International Conference on World Wide Web. New York:ACM Press, 2014: 515-526.
[3]	HUANG X , ZHANG J , LI D ,et al. Knowledge graph embedding based question answering[C]// The Twelfth ACM International Conference. New York:ACM Press, 2019: 105-113.
[4]	ZHANG F , YUAN N J , LIAN D ,et al. Collaborative knowledge base embedding for recommender systems[C]// The 22nd ACM SIGKDD International Conference. New York:ACM Press, 2016: 353-362.
[5]	全拥, 贾焰, 张良 ,等. 在线社交网络个体影响力算法测试与性能评估[J]. 通信学报, 2018,39(10): 1-10.
	QUAN Y , JIA Y , ZHANG L ,et al. Performance analysis and testing of personal influence algorithm in online social networks[J]. Journal on Communications, 2018,39(10): 1-10.
[6]	吴玺煜, 陈启买, 刘海 ,等. 基于知识图谱表示学习的协同过滤推荐算法[J]. 计算机工程, 2018,44(2): 226-232,263.
	WU X Y , CHEN Q M , LIU H ,et al. Collaborative filtering recommendation algorithm based on representation learning of knowledge graph[J]. Computer Engineering, 2018,44(2): 226-232,263.
[7]	SOCHER R , CHEN D , MANNING C D ,et al. Reasoning with neural tensor networks for knowledge base completion[C]// The 26th International Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2013: 926-934.
[8]	黄杨琛, 贾焰, 甘亮 ,等. 基于远程监督的多因子人物关系抽取模型[J]. 通信学报, 2018,39(7): 103-112.
	HUANG Y C , JIA Y , GAN L ,et al. Multi-factor person entity relation extraction model based on distant supervision[J]. Journal on Communications, 2018,39(7): 103-112.
[9]	NIE B , SUN S . Knowledge graph embedding via reasoning over entities,relations,and text[J]. Future Generation Computer Systems, 2018,91: 426-433.
[10]	SUCHANEK F M , KASNECI G , WEIKUM G ,et al. Yago:a core of semantic knowledge[C]// The6th International Conference on World Wide Web. New York:ACM Press, 2007: 697-706.
[11]	LEHMANN J , ISELE R , JAKOB M ,et al. DBPedia——a large-scale,multilingual knowledge base extracted from Wikipedia[J]. Social Work, 2015,6(2): 167-195.
[12]	BOLLACKER K , EVANS C , PARITOSH P ,et al. Freebase:a collaboratively created graph database for structuring human knowledge[C]// The 2008 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2008: 1247-1250.
[13]	赵晓娟, 贾焰, 李爱平 ,等. 多源知识融合技术研究综述[J]. 云南大学学报(自然科学版), 2020,42(3): 459-473.
	ZHAO X J , JIA Y , LI A P ,et al. A survey of the research on multi-source knowledge fusion technology[J]. Journal of Yunnan University:Natural Sciences Edition, 2020,42(3): 459-473.
[14]	KROMPAS D , BAIER S , TRESP V ,et al. Type-constrained representation learning in knowledge graphs[C]// 14th International Semantic Web Conference. Berlin:Springer, 2015: 640-655.
[15]	BANSAL T , JUAN D , RAVI S ,et al. A2N:attending to neighbors for knowledge graph inference[C]// The 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:ACL Press, 2019: 4387-4392.
[16]	SCARSELLI F , GORI M , TSOI A C ,et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009,20(1): 61-80.
[17]	KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks[J]. arXiv Preprint,arXiv:1609.02907, 2016.
[18]	HAMILTON W L , YING R , LESKOVEC J . Inductive representation learning on large graphs[C]// The 31st International Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2017: 1024-1034.
[19]	VELIKOVI P , CUCURULL G , CASANOVA A ,et al. Graph attention networks[J]. arXiv Preprint,arXiv:1710.10903, 2017.
[20]	VASHISHTH S , SANYAL S , NITIN V ,et al. Composition-based multi-relational graph convolutional networks[J]. arXiv Preprint,arXiv:1911.03082, 2019.
[21]	NATHANI D , CHAUHAN J , SHARMA C ,et al. Learning attention-based embeddings for relation prediction in knowledge graphs[C]// The 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:ACL Press, 2019: 4710-4723.
[22]	DAS R , DHULIAWALA S , ZAHEER M ,et al. Go for a walk and arrive at the answer:reasoning over paths in knowledge bases using reinforcement learning[J]. arXiv Preprint,arXiv:1711.05851, 2017.
[23]	WANG X , JI H , SHI C ,et al. Heterogeneous graph attention network[C]// The World Wide Web Conference. New York:ACM Press, 2019: 2022-2032.
[24]	DAI Q N , TU D N , NGUYEN D Q ,et al. A novel embedding model for knowledge base completion based on convolutional neural network[J]. arXiv Preprint,arXiv:1712.02121, 2017.
[25]	BORDES A , USUNIER N , GARCIADURAN A ,et al. Translating embeddings for modeling multi-relational data[C]// The 26th International Conference on Neural Information Processing System. New York:ACM Press, 2013: 2787-2795.
[26]	TOUTANOVA K , CHEN D , PANTEL P ,et al. Representing text for joint embedding of text and knowledge bases[C]// Empirical Methods in Natural Language Processing.[S.n:s.l.], 2015: 1499-1509.
[27]	DETTMERS T , MINERVINI P , STENETORP P ,et al. Convolutional 2D knowledge graph embeddings[J]. arXiv Preprint,arXiv:1707.01476, 2017.
[28]	BORDES A , GLOROT X , WESTON J ,et al. A semantic matching energy function for learning with multi-relational data[J]. Machine Learning, 2014,94(2): 233-259.
[29]	YANG B , YIH W , HE X ,et al. Embedding entities and relations for learning and inference in knowledge bases[J]. arXiv Preprint,arXiv:1412.6575, 2014.
[30]	TROUILLON T , WELBL J , RIEDEL S ,et al. Complex embeddings for simple link prediction[J]. arXiv Preprint,arXiv:1606.06357, 2016.
[31]	KOK S , DOMINGOS P . Statistical predicate invention[C]// The 24th International Conference on Machine Learning. New York:ACM Press, 2007: 433-440.

数据集	实体数量/个	关系数量/个	训练集三元组/个	验证集三元组/个	测试集三元组/个	三元组总数/个
FB15k-237	14 541	237	272 115	17 535	20 466	310 116
WN18RR	40 943	11	86 835	3 034	3 134	93 003

模型	Hits@1	Hits@3	Hits@10	MRR
Complex	0.303	0.434	0.572	0.394
DistMult	0.275	0.417	0.568	0.37
ConvE	0.313	0.457	0.6	0.41
Minerva	0.217	0.329	0.456	0.293
A2N	0.328	0.464	0.608	0.422
RAKGR	0.331	0.476	0.621	0.457

模型	Hits@1	Hits@3	Hits@10	MRR
TransE	0.198	0.376	0.441	0.279
Complex	0.158	0.275	0.428	0.247
DistMult	0.196	0.304	0.444	0.278
ConvE	0.237	0.356	0.501	0.325
ConvKB	0.198	0.324	0.471	0.289
A2N	0.232	0.348	0.486	0.317
RAKGR	0.242	0.371	0.52	0.332

模型	Hits@1	Hits@3	Hits@10	MRR
Complex	0.382	0.433	0.48	0.415
DistMult	0.41	0.441	0.475	0.433
ConvE	0.403	0.452	0.519	0.438
Minerva	0.413	0.456	0.513	0.448
A2N	0.45	0.5	0.55	0.49
RAKGR	0.445	0.49	0.535	0.481

模型	Hits@1	Hits@3	Hits@10	MRR
Complex	0.382	0.433	0.48	0.415
DistMult	0.41	0.441	0.475	0.433
ConvE	0.403	0.452	0.519	0.438
Minerva	0.413	0.456	0.513	0.448
A2N	0.45	0.5	0.55	0.49
RAKGR	0.445	0.49	0.535	0.481

基于层级注意力机制的链接预测模型研究

Research on link prediction model based on hierarchical attention mechanism

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 31

相关文章 1

Metrics

推荐阅读 0