大规模知识图谱预训练模型及电商应用

doi:10.11959/j.issn.2096-0271.2021028

Abstract

Abstract:

In recent years, knowledge graph has been widely applied to organize data in a uniform way and enhance many tasks that require knowledge.For example, it has been widely used in the field of e-commerce.However, such knowledge services usually include tedious data selection and model design for knowledge infusion, which might bring inappropriate results.Thus, to solve this problem, the method of first pre-training then providing knowledge vector service was put forward, and a pre-trained knowledge graph model (PKGM) was proposed for our billionscale e-commerce product knowledge graph, providing item knowledge services in a uniform way for embeddingbased models without accessing triple data in the knowledge graph.PKGM was tested in three knowledge-related tasks including item classification, same item identification, and recommendation.Experimental results show PKGM successfully improves the performance of each task.

Key words: knowledge graph, pre-training, e-commerce

CLC Number:

TP183

Huajun CHEN, Wen ZHANG, Chi-Man WONG, Ganqiang YE, Bo WEN, Wei ZHANG. Large scale pre-trained knowledge graph model and e-commerce application[J]. Big Data Research, 2021, 7(3): 97-115.

Figures/Tables 18

模块	预训练阶段	服务阶段
三元组查询模块	$f_{triple} (h, r, t) = \| \| h + r - t \| \| $	$S_{rel} (h, r) = h + r$
关系查询模块	$f_{rel} (h, r) = \| \| M_{r} h - r \| \|$	$S_{rel} (h, r) = M_{r} h - r$

References 17

[1]	PECHSIRI C , PIRIYAKUL R . Explanation knowledge graph construction through causality extraction from texts[J]. Journal of Computer Science and Technology, 2010,25(5): 1055-1070.
[2]	RADFORD A , NARASIMHAN K , SALIMANS T ,et al. Improving language understanding by generative pre-training[Z]. 2018.
[3]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint, 2018,arXiv:1810.04805.
[4]	YANG Z L , DAI Z H , YANG Y M ,et al. XLNet:generalized autoregressive pretraining for language understanding[J]. arXiv preprint, 2019,arXiv:1906.08237.
[5]	MIKOLOV T , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint, 2013,arXiv:1301.3781.
[6]	BORDES A , USUNIER N , GARCIADURáN A ,et al. Translating embeddings for modeling multi-relational data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS).[S.l.]: Curran Associates Inc., 2013: 2787-2795.
[7]	PARK M Y , HASTIE T . L1-regularization path algorithm for generalized linear models[J]. Journal of the Royal Statistical Society:Series B (Statistical Methodology), 2007,69(4): 659-677.
[8]	JI S X , PAN S R , CAMBRIA E ,et al. A survey on knowledge graphs:Representation,acquisition and applications[J]. arXiv preprint, 2020,arXiv:2002.00388.
[9]	MELO A , PAULHEIM H . Detection of relation assertion errors in knowledge graphs[C]// Proceedings of the 2017 Knowledge Capture Conference. New York:ACM Press, 2017: 1-8.
[10]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[J]. arXiv preprint, 2017,arXiv:1706.03762.
[11]	ZHU Y K , KIROS R , ZEMEL R ,et al. Aligning books and movies:towards story-like visual explanations by watching movies and reading books[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2015: 19-27.
[12]	V?LKEL M , KR?TZSCH M , VRANDECIC D ,et al. Semantic Wikipedia[C]// Proceedings of the 15th International Conference on World Wide Web. New York:ACM Press, 2006: 585-594.
[13]	SUCHANEK F M , KASNECI G , WEIKUM G . YAGO:a core of semantic knowledge[C]// Proceedings of the 16th International Conference on World Wide Web. New York:ACM Press, 2007: 697-706.
[14]	BOLLACKER K , EVANS C , PARITOSH P ,et al. Freebase:a collaboratively created graph database for structuring human knowledge[C]// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2008: 1247-1250.
[15]	MILLER G A , BECKWITH R , FELLBAUM C ,et al. Introduction to WordNet:an on-line lexical database[J]. International Journal of Lexicography, 1990,3(4): 235-244.
[16]	SINOARA R A , CAMACHO-COLLADOS J , ROSSI R G ,et al. Knowledgeenhanced document embeddings for text classification[J]. Knowledge-Based Systems, 2019,163: 955-971.
[17]	HE X N , LIAO L Z , ZHANG H W ,et al. Neural collaborative filtering[C]// Proceedings of the 26th International Conference on World Wide Web.[S.l.:s.n.], 2017: 173-182.

Metrics

Recommended 0

No Suggested Reading articles found!

数据集	#Category	#Train	#Test	#Dev
数量/个	1 293	169 039	36 225	36 223

数据集	模型	Hit@1	Hit@3	Hit@10	AC
dataset-100	BERT	71.03%	84.91%	92.47%	71.52%
	BERT_PKGM-T	71.26%	85.76%	93.07%	72.14%
	BERT_PKGM-R	71.55%	85.43%	92.86%	72.26%
	BERT_PKGM-all	71.64%	85.90%	93.17%	72.19%
dataset-50	BERT	60.98%	78.99%	89.21%	59.06%
	BERT_PKGM-T	61.47%	79.04%	90.08%	62.74%
	BERT_PKGM-R	61.52%	80.09%	90.39%	62.98%
	BERT_PKGM-all	61.54%	79.89%	90.36%	62.71%
dataset-20	BERT	30.26%	46.97%	68.12%	28.90%
	BERT_PKGM-T	30.65%	47.80%	67.40%	29.62%
	BERT_PKGM-R	31.47%	50.69%	69.07%	30.63%
	BERT_PKGM-all	32.09%	50.19%	70.07%	30.91%

对比项	#Train	#Test-C	#Dev-C	# Test-R	#Dev-R
category-1/个	4 731	1 014	1 013	513	497
category-2/个	2 424	520	519	268	278
category-3/个	3 968	852	850	417	440

数据集	模型	Hit@1	Hit@3	Hit@10
category-1	BERT	65.06%	76.06%	86.68%
	BERT_PKGM-all	64.75%	77.50%	87.43%
category-2	BERT	65.86%	78.07%	87.59%
	BERT_PKGM-all	66.13%	78.19%	87.96%
category-3	BERT	49.64%	66.18%	82.37%
	BERT_PKGM-all	50.60%	67.14%	83.45%

模型	category-1	category-2	category-3
BERT	88.94%	89.31%	86.94%
BERT_PKGM-T	88.65%	89.89%	87.88%
BERT_PKGM-R	89.09%	89.60%	87.88%
BERT_PKGM-all	89.15%	90.08%	88.13%

Large scale pre-trained knowledge graph model and e-commerce application

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 17

Related Articles 15

Metrics

Recommended 0

数据集	#Items	#Users	#Interactions
商品推荐任务	37 847	29 015	443 425

模型	HR@1	HR@3	HR@5	HR@10	HR@30	NDCG@1	NDCG@3	NDCG@5	NDCG@10	NDCG@30
NCF	27.94%	44.26%	52.16%	62.88%	81.26%	0.279 4	0.374 4	0.406 9	0.441 5	0.485 3
NCF_PKGM-T	27.96%	44.83%	52.43%	63.51%	81.62%	0.279 6	0.377 8	0.409 1	0.444 9	0.488 0
NCF_PKGM-R	31.01%	47.99%	56.10%	66.98%	84.73%	0.310 1	0.409 1	0.442 4	0.477 7	0.520 0
NCF_PKGM-all	30.76%	47.92%	55.60%	66.84%	84.71%	0.307 6	0.407 9	0.439 5	0.475 8	0.518 5

[1]	Yuqi ZHANG, Xiaowen HUANG, Jitao SANG. Knowledge-enhanced policy-guided interactive reinforcement recommendation system [J]. Big Data Research, 2022, 8(5): 88-105.
[2]	Jingye SHI, Pan LI. Key technologies and application exploration of aerospace big data in the construction of new smart city [J]. Big Data Research, 2022, 8(2): 120-133.
[3]	Ling TAN, Haihong E, Zemin KUANG, Meina SONG, Yu LIU, Zhengyu CHEN, Xiaoxuan XIE, Jundi LI, Jiawei FAN, Qingchuan WANG, Xiaoyang KANG. Key technologies and research progress of medical knowledge graph construction [J]. Big Data Research, 2021, 7(4): 80-104.
[4]	Qingxia LIU, Junyou LI, Gong CHENG. An interpretive evaluation of entity summarization system [J]. Big Data Research, 2021, 7(3): 15-29.
[5]	Yuming SHEN, Jianfeng DU. Temporal knowledge graph completion:methods and progress [J]. Big Data Research, 2021, 7(3): 30-41.
[6]	Huifang DU, Haofen WANG, Yinghui SHI, Meng WANG. Progress, challenges and research trends of reasoning in multi-hop knowledge graph based question answering [J]. Big Data Research, 2021, 7(3): 60-79.
[7]	Zhilei HU, Xiaolong JIN, Jianyun CHEN, Guanli HUANG. Construction, reasoning and applications of event graphs [J]. Big Data Research, 2021, 7(3): 80-96.
[8]	Qiang CHEN, Shiya DAI. Recognition method of accounting fraud risk based on financial knowledge graph [J]. Big Data Research, 2021, 7(3): 116-129.
[9]	Bing XIE, Xin PENG, Gang YIN, Xuandong LI, Jun WEI, Hailong SUN. Big data based intelligent software development methodology and environment [J]. Big Data Research, 2021, 7(1): 3-21.
[10]	Yanzhen ZOU, Min WANG, Bing XIE, Zeqi LIN. Software knowledge graph construction and Q＆amp;A technology based on big data [J]. Big Data Research, 2021, 7(1): 22-36.
[11]	Wei CHEN, Hongjie YE, Jiahong ZHOU, Jun WEI. An approach to automatically building Docker images by using domain knowledge [J]. Big Data Research, 2021, 7(1): 64-75.
[12]	Jian ZHANG, Xiangxin MENG, Hailong SUN, Xu WANG, Xudong LIU. Data driven intelligent collaboration of software developers [J]. Big Data Research, 2021, 7(1): 76-93.
[13]	Cheng CHEN, Yueguo CHEN, Chen LIU, Xiaotong LYU, Xiaoyong DU. Constructing and analyzing intention knowledge graphs [J]. Big Data Research, 2020, 6(2): 57-68.
[14]	Genlin ZANG, Yaqiang WANG, Qingrong WU, Chunli ZHAN, Yi LI. Model and construction method of the ontology of knowledge graph of smart city [J]. Big Data Research, 2020, 6(2): 96-106.
[15]	Ying LIANG, Wei ZHANG, Zhidong YU, Hongzhou SHI. Applications of academic big data in the process of science and technology management [J]. Big Data Research, 2019, 5(5): 3-15.