基于双曲空间图嵌入的科研热点预测

doi:10.11959/j.issn.2096-0271.2022041

摘要/Abstract

摘要：

预测科研热点可以有效地开展科学研究和更好地分配科学资源。数据挖掘和机器学习算法已经被广泛应用到科研热点预测中，比如基于论文文本内容的主题模型建模和挖掘论文被引频次的算法等。提出一种新的将关键词信息嵌入双曲空间的双曲空间关键词图嵌入（PKGM）算法，利用关键词和它们之间的关系构建一个关键词网络，通过计算双曲空间中两个节点的距离来判别两个节点之间存在边的概率，从而对科研热点进行预测。该算法与7个基准算法的实验比较结果显示，PKGM算法与效果最好的欧氏空间嵌入算法相比有7.3%的AUROC和5.8%的AP提升；与双曲图神经网络算法相比，有10.8%的AUROC和7.2%的AP提升。这显示了PKGM算法的有效性。

关键词: 科研热点, 双曲空间, 庞加莱模型, 图嵌入, 关键词网络, 长尾效应

Abstract:

Scientific topic prediction is central to scientific research and can substantially advance the allocation of scientific resources.Machine learning and data mining approaches have been widely applied to scientific topic prediction, including paper content-based topic model and citation prediction models.A novel scientific topic prediction algorithm PKGM (Poincare keywords graph embedding) was proposed, which utilized keywords and their relations to build a keyword network, and calculated the distance between two nodes in this network to predict the probability that an edge existed.The result of comparing PKGM with seven baselines showed that PKGM obtained a 7.3% improvement by using AUROC and 5.8% improving by using AP in comparison to the best method in Euclidean space, and 10.8% improvement by using AUROC and 7.2% improving by using AP over the best approach in hyperbolic space.The results demonstrated the effectiveness of PKGM.

Key words: scientific topic, hyperbolic space, Poincare's model, graph embedding, keywords network, long tail effect

中图分类号:

TP18

戴筠. 基于双曲空间图嵌入的科研热点预测[J]. 大数据, 2022, 8(6): 94-104.

Jun DAI. Emerging scientific topic prediction based on Poincare graph embedding[J]. Big Data Research, 2022, 8(6): 94-104.

图/表 8

图1

图2

表1

图3

表2

表3

图4

图5

参考文献 33

[1]	YANN L , YOSHUA B , GEOFFREY H . Deep learning[J]. Nature, 2015,521: 436-444.
[2]	ATTARDI G , . DeepNL:a deep learning NLP pipeline[C]// Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing.[S.l.:s.n.], 2015: 109-115.
[3]	VOULODIMOS A , DOULAMIS N , DOULAMIS A ,et al. Deep learning for computer vision:a brief review[J]. Computational Intelligence and Neuroscience, 2018: 1-13.
[4]	WANG C , BLEI D M . Collaborative topic modeling for recommending scientific articles[C]// Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.[S.l.:s.n.], 2011: 448-456.
[5]	ZHANG S , TONG H H , XU J J ,et al. Graph convolutional networks:a comprehensive review[J]. Computational Social Networks, 2019,6(11): 1-23.
[6]	WALLACH H M , . Topic modeling:beyond bag-of-words[C]// Proceedings of the 23rd International Conference on Machine Learning.[S.l.:s.n.], 2006: 977-984.
[7]	BHAT H S , HUANG L H , RODRIGUEZ S ,et al. Citation prediction using diverse features[C]// Proceedings of 2015 IEEE International Conference on Data Mining Workshop. Piscataway:IEEE Press, 2015: 589-596.
[8]	MORADI B , PARENT M C , WEIS A S ,et al. Mapping the travels of intersectionality scholarship:a citation network analysis[J]. Psychology of Women Quarterly, 2020,44(2): 151-169.
[9]	LIU L S , YU D J , WANG D J ,et al. Citation count prediction based on neural hawkes model[J]. IEICE Transactions on Information and Systems, 2020,103(11): 2379-2388.
[10]	JEONG C , JANG S , PARK E ,et al. A context-aware citation recommendation model with BERT and graph convolutional networks[J]. Scientometrics, 2020,124(3): 1907-1922.
[11]	GROVER A , LESKOVEC J . node2vec:scalable feature learning for networks[C]// Proceedings of the 22nd SIGKDD Conference on Knowledge Discovery and Data Mining.[S.l.:s.n.], 2016: 855-864.
[12]	CHAMI I , YING R , RE C ,et al. Hyperbolic graph convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2019,32: 4869-4880.
[13]	LEE J , LEE J N , SHIN H . The long tail or the short tail:the category-specific impact of eWOM on sales distribution[J]. Decision Support Systems, 2011,51(3): 466-479.
[14]	YANG C , LIU Z , ZHAO D ,et al. Network representation learning with rich text information[C]// Proceedings of the 24th International Joint Conference on Artificial Intelligence.[S.l.:s.n.], 2015: 2111-2117.
[15]	YUE X , WANG Z , HUANG J G ,et al. Graph embedding on biomedical networks:methods,applications and evaluations[J]. Bioinformatics, 2020,36(4): 1241-1251.
[16]	ASHOOR H , CHEN X W , ROSIKIEWICZ W ,et al. Graph embedding and unsupervised learning predict genomic sub-compartments from HiC chromatin interaction data[J]. Nature Communications, 2020,11:1173.
[17]	PEROZZI B , AL-RFOU R , SKIENA S . DeepWalk: online learning of social representations[C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2014: 701-710.
[18]	TANG J , QU M , WANG M Z ,et al. LINE:large-scale information network embedding[C]// Proceedings of the 24th International Conference on World Wide Web.[S.l.:s.n.], 2015: 1067-1077.
[19]	KIPF T N , WELLING M . Semisupervised classification with graph convolutional networks[C]// Proceedings of the International Conference on Learning Representation.[S.l.:s.n.], 2017: 1-8.
[20]	GILMER J , SCHOENHOLZ S S , RILEY P F ,et al. Neural message passing for quantum chemistry[C]// Proceedings of the 34th International Conference on Machine Learning.[S.l.:s.n.], 2017: 1263-1272.
[21]	BOSCAINI D , MASCI J , RODOLA E ,et al. Learning shape correspondence with anisotropic convolutional neural networks[C]// Proceedings of the 30th Conference on Neural Information Processing Systems.[S.l.:s.n.], 2016: 3189-3197.
[22]	MONTI F , BOSCAINI D , MASCI J ,et al. Geometric deep learning on graphs and manifolds using mixture model CNNs[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 5115-5124.
[23]	KAWAMOTO T , TSUBAKI M , OBUCHI T . Mean-field theory of graph neural networks in graph partitioning[C]// Proceedings of the 32nd International Coference on Neural Information Processing Systems.Red Hook:Curran Associates Inc. , 2018: 4366-4376.
[24]	WANG X , JI H , SHI C ,et al. Heterogeneous graph attention network[C]// Proceedings of the 28th International Conference on World Wide Web.[S.l.:s.n.], 2019: 2022-2032.
[25]	VELICKOVIC P , CUCURULL G , CASANOVA A ,et al. Graph attention networks[C]// Proceedings of the International Conference on Learning Representations 2018.[S.l.:s.n.], 2018: 1-8.
[26]	TIFREA A , BECIGNEUL G , GANEA O E . PoincaréGloVe:hyperbolic word embeddings[C]// Proceedings of the International Conference on Learning Representations 2019.[S.l.:s.n.], 2019: 1-8.
[27]	DHINGRA B , SHALLUE C J , NOROUZI M ,et al. Embedding text in hyperbolic space[C]// Proceedings of the Association for Computational Linguistics.[S.l.:s.n.], 2018: 59-69.
[28]	GANEA O , BECIGNEUL G , HOFMANN T . Hyperbolic neural networks[C]// Proceedings of the 32nd Conference on Neural Information Processing Systems.[S.l.:s.n.], 2018: 5345-5355.
[29]	LIU Q , NICKEL M , KIELA D . Hyperbolic graph neural networks[C]// Proceedings of the 33rd Conference on Neural Information Processing Systems.[S.l.:s.n.], 2019: 8230-8241.
[30]	ASLAN S , KAYA M . Topic recommendation for authors as a link prediction problem[J]. Future Generation Computer Systems, 2018,89: 249-264.
[31]	SONG W Z , CHEN H X , LIU X Y ,et al. Hyperbolic node embedding for signed networks[J]. Neurocomputing, 2021,421: 329-339.
[32]	NICKEL M , KIELA D . Poincaré embeddings for learning hierarchical representations[C]// Proceedings of the 31st Conference on Neural Information Processing Systems.[S.l.:s.n.], 2017: 6338-6347.
[33]	WEI C H , KAO H Y , LU Z Y . PubTator:a web-based text mining tool for assisting biocuration[J]. Nucleic Acids Research, 2013,41(W1): 518-522.

实验	测试集/条	训练集/条	验证集/条
实验1	1 897	16 130	949
实验2	5 692	11 386	1 898

算法	实验1				实验2
算法	AUROC	增加	AP	增加	AUROC	增加	AP	增加
PKGM	0.8822	-	0.8906	-	0.8751	-	0.8857	-
Euclidean	0.8180	7.3%	0.8389	5.8%	0.8062	7.9%	0.8276	6.6%
MLP	0.4989	43.4%	0.4995	43.9%	0.5003	42.8%	0.5001	43.5%
GCN	0.7222	18.1%	0.7448	16.4%	0.7406	15.4%	0.7329	17.3%
GAT	0.7590	14.0%	0.7705	13.5%	0.7575	13.4%	0.7674	13.4%

算法	实验1				实验2
算法	AUROC	增加	AP	增加	AUROC	增加	AP	增加
PKGM	0.8822	-	0.8906	-	0.8751	-	0.8857	-
HNN	0.6596	25.2%	0.6466	27.4%	0.6817	22.1%	0.6645	25.0%
HGCN	0.7764	12.0%	0.7930	11.0%	0.7750	11.4%	0.7943	10.3%
HGNN	0.7865	10.8%	0.8264	7.2%	0.7787	11.0%	0.8162	7.8%