基于层级注意力机制的链接预测模型研究

doi:10.11959/j.issn.1000-436x.2021057

Abstract

Abstract:

In order to solve the problem that the existing graph attention mechanism tends to cause attention distribution to certain relations with high frequency when performing link prediction related tasks, a new link prediction model based on hierarchical attention mechanism was proposed.In the link prediction task, a hierarchical attention mechanism was designed to give different attention to the relationships of different relationship types connected to a given entity in the knowledge graph according to the relationship in the prediction task.While the characteristics of multi-hop neighbor entities were pay attention to, the relationship characteristics was pay more attention to find the relationship type that matches the target relationship.Through comparison experiments with the mainstream models on multiple benchmark data sets, the results show that the performance of the model is better than the mainstream models and has good robustness.

Key words: hierarchical attention mechanism, link prediction, knowledge graph embedding

CLC Number:

TP391

Xiaojuan ZHAO, Yan JIA, Aiping LI, Kai CHEN. Research on link prediction model based on hierarchical attention mechanism[J]. Journal on Communications, 2021, 42(3): 36-44.

Figures/Tables 9

References 31

[1]	BAO J , DUAN N , YAN Z ,et al. Constraint-based question answering with knowledge graph[C]// International Conference on Computational Linguistics. Saarland:DBLP, 2016: 2503-2514.
[2]	WEST R , GABRILOVICH E , MURPHY K ,et al. Knowledge base completion via search-based question answering[C]// The 23rd International Conference on World Wide Web. New York:ACM Press, 2014: 515-526.
[3]	HUANG X , ZHANG J , LI D ,et al. Knowledge graph embedding based question answering[C]// The Twelfth ACM International Conference. New York:ACM Press, 2019: 105-113.
[4]	ZHANG F , YUAN N J , LIAN D ,et al. Collaborative knowledge base embedding for recommender systems[C]// The 22nd ACM SIGKDD International Conference. New York:ACM Press, 2016: 353-362.
[5]	全拥, 贾焰, 张良 ,等. 在线社交网络个体影响力算法测试与性能评估[J]. 通信学报, 2018,39(10): 1-10.
	QUAN Y , JIA Y , ZHANG L ,et al. Performance analysis and testing of personal influence algorithm in online social networks[J]. Journal on Communications, 2018,39(10): 1-10.
[6]	吴玺煜, 陈启买, 刘海 ,等. 基于知识图谱表示学习的协同过滤推荐算法[J]. 计算机工程, 2018,44(2): 226-232,263.
	WU X Y , CHEN Q M , LIU H ,et al. Collaborative filtering recommendation algorithm based on representation learning of knowledge graph[J]. Computer Engineering, 2018,44(2): 226-232,263.
[7]	SOCHER R , CHEN D , MANNING C D ,et al. Reasoning with neural tensor networks for knowledge base completion[C]// The 26th International Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2013: 926-934.
[8]	黄杨琛, 贾焰, 甘亮 ,等. 基于远程监督的多因子人物关系抽取模型[J]. 通信学报, 2018,39(7): 103-112.
	HUANG Y C , JIA Y , GAN L ,et al. Multi-factor person entity relation extraction model based on distant supervision[J]. Journal on Communications, 2018,39(7): 103-112.
[9]	NIE B , SUN S . Knowledge graph embedding via reasoning over entities,relations,and text[J]. Future Generation Computer Systems, 2018,91: 426-433.
[10]	SUCHANEK F M , KASNECI G , WEIKUM G ,et al. Yago:a core of semantic knowledge[C]// The6th International Conference on World Wide Web. New York:ACM Press, 2007: 697-706.
[11]	LEHMANN J , ISELE R , JAKOB M ,et al. DBPedia——a large-scale,multilingual knowledge base extracted from Wikipedia[J]. Social Work, 2015,6(2): 167-195.
[12]	BOLLACKER K , EVANS C , PARITOSH P ,et al. Freebase:a collaboratively created graph database for structuring human knowledge[C]// The 2008 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2008: 1247-1250.
[13]	赵晓娟, 贾焰, 李爱平 ,等. 多源知识融合技术研究综述[J]. 云南大学学报(自然科学版), 2020,42(3): 459-473.
	ZHAO X J , JIA Y , LI A P ,et al. A survey of the research on multi-source knowledge fusion technology[J]. Journal of Yunnan University:Natural Sciences Edition, 2020,42(3): 459-473.
[14]	KROMPAS D , BAIER S , TRESP V ,et al. Type-constrained representation learning in knowledge graphs[C]// 14th International Semantic Web Conference. Berlin:Springer, 2015: 640-655.
[15]	BANSAL T , JUAN D , RAVI S ,et al. A2N:attending to neighbors for knowledge graph inference[C]// The 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:ACL Press, 2019: 4387-4392.
[16]	SCARSELLI F , GORI M , TSOI A C ,et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009,20(1): 61-80.
[17]	KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks[J]. arXiv Preprint,arXiv:1609.02907, 2016.
[18]	HAMILTON W L , YING R , LESKOVEC J . Inductive representation learning on large graphs[C]// The 31st International Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2017: 1024-1034.
[19]	VELIKOVI P , CUCURULL G , CASANOVA A ,et al. Graph attention networks[J]. arXiv Preprint,arXiv:1710.10903, 2017.
[20]	VASHISHTH S , SANYAL S , NITIN V ,et al. Composition-based multi-relational graph convolutional networks[J]. arXiv Preprint,arXiv:1911.03082, 2019.
[21]	NATHANI D , CHAUHAN J , SHARMA C ,et al. Learning attention-based embeddings for relation prediction in knowledge graphs[C]// The 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:ACL Press, 2019: 4710-4723.
[22]	DAS R , DHULIAWALA S , ZAHEER M ,et al. Go for a walk and arrive at the answer:reasoning over paths in knowledge bases using reinforcement learning[J]. arXiv Preprint,arXiv:1711.05851, 2017.
[23]	WANG X , JI H , SHI C ,et al. Heterogeneous graph attention network[C]// The World Wide Web Conference. New York:ACM Press, 2019: 2022-2032.
[24]	DAI Q N , TU D N , NGUYEN D Q ,et al. A novel embedding model for knowledge base completion based on convolutional neural network[J]. arXiv Preprint,arXiv:1712.02121, 2017.
[25]	BORDES A , USUNIER N , GARCIADURAN A ,et al. Translating embeddings for modeling multi-relational data[C]// The 26th International Conference on Neural Information Processing System. New York:ACM Press, 2013: 2787-2795.
[26]	TOUTANOVA K , CHEN D , PANTEL P ,et al. Representing text for joint embedding of text and knowledge bases[C]// Empirical Methods in Natural Language Processing.[S.n:s.l.], 2015: 1499-1509.
[27]	DETTMERS T , MINERVINI P , STENETORP P ,et al. Convolutional 2D knowledge graph embeddings[J]. arXiv Preprint,arXiv:1707.01476, 2017.
[28]	BORDES A , GLOROT X , WESTON J ,et al. A semantic matching energy function for learning with multi-relational data[J]. Machine Learning, 2014,94(2): 233-259.
[29]	YANG B , YIH W , HE X ,et al. Embedding entities and relations for learning and inference in knowledge bases[J]. arXiv Preprint,arXiv:1412.6575, 2014.
[30]	TROUILLON T , WELBL J , RIEDEL S ,et al. Complex embeddings for simple link prediction[J]. arXiv Preprint,arXiv:1606.06357, 2016.
[31]	KOK S , DOMINGOS P . Statistical predicate invention[C]// The 24th International Conference on Machine Learning. New York:ACM Press, 2007: 433-440.

Metrics

Recommended 0

No Suggested Reading articles found!

数据集	实体数量/个	关系数量/个	训练集三元组/个	验证集三元组/个	测试集三元组/个	三元组总数/个
FB15k-237	14 541	237	272 115	17 535	20 466	310 116
WN18RR	40 943	11	86 835	3 034	3 134	93 003

模型	Hits@1	Hits@3	Hits@10	MRR
Complex	0.303	0.434	0.572	0.394
DistMult	0.275	0.417	0.568	0.37
ConvE	0.313	0.457	0.6	0.41
Minerva	0.217	0.329	0.456	0.293
A2N	0.328	0.464	0.608	0.422
RAKGR	0.331	0.476	0.621	0.457

模型	Hits@1	Hits@3	Hits@10	MRR
TransE	0.198	0.376	0.441	0.279
Complex	0.158	0.275	0.428	0.247
DistMult	0.196	0.304	0.444	0.278
ConvE	0.237	0.356	0.501	0.325
ConvKB	0.198	0.324	0.471	0.289
A2N	0.232	0.348	0.486	0.317
RAKGR	0.242	0.371	0.52	0.332

模型	Hits@1	Hits@3	Hits@10	MRR
Complex	0.382	0.433	0.48	0.415
DistMult	0.41	0.441	0.475	0.433
ConvE	0.403	0.452	0.519	0.438
Minerva	0.413	0.456	0.513	0.448
A2N	0.45	0.5	0.55	0.49
RAKGR	0.445	0.49	0.535	0.481

模型	Hits@1	Hits@3	Hits@10	MRR
Complex	0.382	0.433	0.48	0.415
DistMult	0.41	0.441	0.475	0.433
ConvE	0.403	0.452	0.519	0.438
Minerva	0.413	0.456	0.513	0.448
A2N	0.45	0.5	0.55	0.49
RAKGR	0.445	0.49	0.535	0.481

Research on link prediction model based on hierarchical attention mechanism

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 31

Related Articles 8

Metrics

Recommended 0

[1]	Rong QIAN, Jianting XU, Kejun ZHANG, Hongyu DONG, Fangyuan XING. Research on HMM based link prediction method in heterogeneous network [J]. Journal on Communications, 2022, 43(5): 214-225.
[2]	Qiuyang GU, Bao WU, Renyong CHI. Link prediction method based on the similarity of high path [J]. Journal on Communications, 2021, 42(7): 61-69.
[3]	Jian SHU, Qining WANG, Linlan LIU. UAV ad hoc network link prediction based on deep graph embedding [J]. Journal on Communications, 2021, 42(7): 137-149.
[4]	Yiteng WU,Hongtao YU,Ruiyang HUANG,Huawei LI. Theoretical limit of link prediction using a combination method [J]. Journal on Communications, 2020, 41(6): 34-50.
[5]	Shuxin LIU,Xing LI,Hongchang CHEN,Kai WANG. Link prediction method based on matching degree of resource transmission for complex network [J]. Journal on Communications, 2020, 41(6): 70-79.
[6]	Qiuyang GU, Chunhua JU, Gongxing WU. Social network link prediction method based on subgraph evolution and improved ant colony optimization algorithm [J]. Journal on Communications, 2020, 41(12): 21-35.
[7]	Zu-feng WU,Qi LIANG,Qiao LIU,Zhi-guang QIN. Modified link prediction algorithm based on AdaBoost [J]. Journal on Communications, 2014, 35(3): 116-123.
[8]	. Modified link prediction algorithm based on AdaBoost [J]. Journal on Communications, 2014, 35(3): 13-123.