大规模知识图谱预训练模型及电商应用

doi:10.11959/j.issn.2096-0271.2021028

大数据 ›› 2021, Vol. 7 ›› Issue (3): 97-115.doi: 10.11959/j.issn.2096-0271.2021028

所属专题：知识图谱

• 专题：基于大数据的知识图谱及其应用 • 上一篇下一篇

大规模知识图谱预训练模型及电商应用

陈华钧¹^,², 张文³, 黄志文⁴, 叶橄强¹, 文博¹, 张伟²^,⁴

¹ 浙江大学计算机科学与技术学院，浙江杭州 310007
² 阿里巴巴-浙江大学前沿技术联合研究中心，浙江杭州 311121
³ 浙江大学软件学院，浙江杭州 310007
⁴ 阿里巴巴集团，浙江杭州 311121

出版日期:2021-05-15 发布日期:2021-05-01
作者简介:陈华钧（1978- ），男，浙江大学计算机科学与技术学院教授，主要研究方向为知识图谱、自然语言处理、大数据系统。
张文（1992- ），女，博士，浙江大学软件学院助理研究员，主要研究方向为知识图谱、知识表示和知识推理。
黄志文（1993- ），男，阿里巴巴集团商品知识图谱团队算法工程师，主要研究方向为深度学习和知识图谱。
叶橄强（1996- ），男，浙江大学计算机科学与技术学院硕士生，主要研究方向为知识图谱表示学习和预训练。
文博（1994- ），男，浙江大学计算机科学与技术学院硕士生，主要研究方向为知识图谱和推荐计算。
张伟（1983- ），男，博士，阿里巴巴集团资深算法专家，主要研究方向为自然语言处理和知识图谱。
基金资助:
国家自然科学基金资助项目(91846204);国家自然科学基金资助项目(U19B2027)

Large scale pre-trained knowledge graph model and e-commerce application

Huajun CHEN¹^,², Wen ZHANG³, Chi-Man WONG⁴, Ganqiang YE¹, Bo WEN¹, Wei ZHANG²^,⁴

¹ College of Computer Science and Technology, Zhejiang University, Hangzhou 310007, China
² Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies, Hangzhou 311121, China
³ School of Software Technology, Zhejiang University, Hangzhou 310007, China
⁴ Alibaba Group, Hangzhou 311121, China

Online:2021-05-15 Published:2021-05-01
Supported by:
The National Natural Science Foundation of China(91846204);The National Natural Science Foundation of China(U19B2027)

摘要/Abstract

摘要：

近年来，知识图谱因具有以统一的方式组织数据等优势，被广泛应用于许多需要知识的任务，并且在电子商务领域大放光彩。然而知识服务通常需要烦琐的数据选择和知识注入模型的设计，这会给业务带来不良影响。为了更好地解决这一问题，提出了“预训练+知识向量服务”的模式，并设计了知识图谱预训练模型（PKGM），在不直接访问商品知识图谱中三元组数据的情况下，以知识向量的方式为下游任务提供知识图谱服务。在商品分类、同款商品识别和商品推荐等知识图谱下游任务中进行测试，实验结果表明，知识图谱预训练模型能够有效地提高每个任务的性能。

关键词: 知识图谱, 预训练, 电商

Abstract:

In recent years, knowledge graph has been widely applied to organize data in a uniform way and enhance many tasks that require knowledge.For example, it has been widely used in the field of e-commerce.However, such knowledge services usually include tedious data selection and model design for knowledge infusion, which might bring inappropriate results.Thus, to solve this problem, the method of first pre-training then providing knowledge vector service was put forward, and a pre-trained knowledge graph model (PKGM) was proposed for our billionscale e-commerce product knowledge graph, providing item knowledge services in a uniform way for embeddingbased models without accessing triple data in the knowledge graph.PKGM was tested in three knowledge-related tasks including item classification, same item identification, and recommendation.Experimental results show PKGM successfully improves the performance of each task.

Key words: knowledge graph, pre-training, e-commerce

中图分类号:

TP183

陈华钧, 张文, 黄志文, 叶橄强, 文博, 张伟. 大规模知识图谱预训练模型及电商应用[J]. 大数据, 2021, 7(3): 97-115.

Huajun CHEN, Wen ZHANG, Chi-Man WONG, Ganqiang YE, Bo WEN, Wei ZHANG. Large scale pre-trained knowledge graph model and e-commerce application[J]. Big Data Research, 2021, 7(3): 97-115.

图/表 18

图1

图2

图3

表1

知识图谱静态预训练模型的预训练阶段和服务阶段的函数"

模块	预训练阶段	服务阶段
三元组查询模块	$f_{triple} (h, r, t) = \| \| h + r - t \| \| $	$S_{rel} (h, r) = h + r$
关系查询模块	$f_{rel} (h, r) = \| \| M_{r} h - r \| \|$	$S_{rel} (h, r) = M_{r} h - r$

表1

图4

图5

图6

图7

图8

表2

表3

图9

表4

表5

表6

图10

表7

表8

参考文献 17

[1]	PECHSIRI C , PIRIYAKUL R . Explanation knowledge graph construction through causality extraction from texts[J]. Journal of Computer Science and Technology, 2010,25(5): 1055-1070.
[2]	RADFORD A , NARASIMHAN K , SALIMANS T ,et al. Improving language understanding by generative pre-training[Z]. 2018.
[3]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint, 2018,arXiv:1810.04805.
[4]	YANG Z L , DAI Z H , YANG Y M ,et al. XLNet:generalized autoregressive pretraining for language understanding[J]. arXiv preprint, 2019,arXiv:1906.08237.
[5]	MIKOLOV T , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint, 2013,arXiv:1301.3781.
[6]	BORDES A , USUNIER N , GARCIADURáN A ,et al. Translating embeddings for modeling multi-relational data[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS).[S.l.]: Curran Associates Inc., 2013: 2787-2795.
[7]	PARK M Y , HASTIE T . L1-regularization path algorithm for generalized linear models[J]. Journal of the Royal Statistical Society:Series B (Statistical Methodology), 2007,69(4): 659-677.
[8]	JI S X , PAN S R , CAMBRIA E ,et al. A survey on knowledge graphs:Representation,acquisition and applications[J]. arXiv preprint, 2020,arXiv:2002.00388.
[9]	MELO A , PAULHEIM H . Detection of relation assertion errors in knowledge graphs[C]// Proceedings of the 2017 Knowledge Capture Conference. New York:ACM Press, 2017: 1-8.
[10]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[J]. arXiv preprint, 2017,arXiv:1706.03762.
[11]	ZHU Y K , KIROS R , ZEMEL R ,et al. Aligning books and movies:towards story-like visual explanations by watching movies and reading books[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2015: 19-27.
[12]	V?LKEL M , KR?TZSCH M , VRANDECIC D ,et al. Semantic Wikipedia[C]// Proceedings of the 15th International Conference on World Wide Web. New York:ACM Press, 2006: 585-594.
[13]	SUCHANEK F M , KASNECI G , WEIKUM G . YAGO:a core of semantic knowledge[C]// Proceedings of the 16th International Conference on World Wide Web. New York:ACM Press, 2007: 697-706.
[14]	BOLLACKER K , EVANS C , PARITOSH P ,et al. Freebase:a collaboratively created graph database for structuring human knowledge[C]// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2008: 1247-1250.
[15]	MILLER G A , BECKWITH R , FELLBAUM C ,et al. Introduction to WordNet:an on-line lexical database[J]. International Journal of Lexicography, 1990,3(4): 235-244.
[16]	SINOARA R A , CAMACHO-COLLADOS J , ROSSI R G ,et al. Knowledgeenhanced document embeddings for text classification[J]. Knowledge-Based Systems, 2019,163: 955-971.
[17]	HE X N , LIAO L Z , ZHANG H W ,et al. Neural collaborative filtering[C]// Proceedings of the 26th International Conference on World Wide Web.[S.l.:s.n.], 2017: 173-182.

数据集	#Category	#Train	#Test	#Dev
数量/个	1 293	169 039	36 225	36 223

数据集	模型	Hit@1	Hit@3	Hit@10	AC
dataset-100	BERT	71.03%	84.91%	92.47%	71.52%
	BERT_PKGM-T	71.26%	85.76%	93.07%	72.14%
	BERT_PKGM-R	71.55%	85.43%	92.86%	72.26%
	BERT_PKGM-all	71.64%	85.90%	93.17%	72.19%
dataset-50	BERT	60.98%	78.99%	89.21%	59.06%
	BERT_PKGM-T	61.47%	79.04%	90.08%	62.74%
	BERT_PKGM-R	61.52%	80.09%	90.39%	62.98%
	BERT_PKGM-all	61.54%	79.89%	90.36%	62.71%
dataset-20	BERT	30.26%	46.97%	68.12%	28.90%
	BERT_PKGM-T	30.65%	47.80%	67.40%	29.62%
	BERT_PKGM-R	31.47%	50.69%	69.07%	30.63%
	BERT_PKGM-all	32.09%	50.19%	70.07%	30.91%

对比项	#Train	#Test-C	#Dev-C	# Test-R	#Dev-R
category-1/个	4 731	1 014	1 013	513	497
category-2/个	2 424	520	519	268	278
category-3/个	3 968	852	850	417	440

数据集	模型	Hit@1	Hit@3	Hit@10
category-1	BERT	65.06%	76.06%	86.68%
	BERT_PKGM-all	64.75%	77.50%	87.43%
category-2	BERT	65.86%	78.07%	87.59%
	BERT_PKGM-all	66.13%	78.19%	87.96%
category-3	BERT	49.64%	66.18%	82.37%
	BERT_PKGM-all	50.60%	67.14%	83.45%

模型	category-1	category-2	category-3
BERT	88.94%	89.31%	86.94%
BERT_PKGM-T	88.65%	89.89%	87.88%
BERT_PKGM-R	89.09%	89.60%	87.88%
BERT_PKGM-all	89.15%	90.08%	88.13%

大规模知识图谱预训练模型及电商应用

Large scale pre-trained knowledge graph model and e-commerce application

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 17

相关文章 15

Metrics

推荐阅读 0

数据集	#Items	#Users	#Interactions
商品推荐任务	37 847	29 015	443 425

模型	HR@1	HR@3	HR@5	HR@10	HR@30	NDCG@1	NDCG@3	NDCG@5	NDCG@10	NDCG@30
NCF	27.94%	44.26%	52.16%	62.88%	81.26%	0.279 4	0.374 4	0.406 9	0.441 5	0.485 3
NCF_PKGM-T	27.96%	44.83%	52.43%	63.51%	81.62%	0.279 6	0.377 8	0.409 1	0.444 9	0.488 0
NCF_PKGM-R	31.01%	47.99%	56.10%	66.98%	84.73%	0.310 1	0.409 1	0.442 4	0.477 7	0.520 0
NCF_PKGM-all	30.76%	47.92%	55.60%	66.84%	84.71%	0.307 6	0.407 9	0.439 5	0.475 8	0.518 5

[1]	崔雨萌, 王靖亚, 闫尚义, 陶知众. 基于深度学习的警情记录关键信息自动抽取[J]. 大数据, 2022, 8(6): 127-142.
[2]	韩立帆, 季紫荆, 陈子睿, 王鑫. 数字人文视域下面向历史古籍的信息抽取方法研究[J]. 大数据, 2022, 8(6): 26-39.
[3]	关海山, 郑玉龙, 魏笔凡, 张泽民, 岳浩, 师斌, 董博. 税收优惠政策关键要素抽取与可视化分析[J]. 大数据, 2022, 8(5): 106-123.
[4]	张宇奇, 黄晓雯, 桑基韬. 知识增强策略引导的交互式强化推荐系统[J]. 大数据, 2022, 8(5): 88-105.
[5]	史经业, 李攀. 空天大数据在新型智慧城市建设中的关键技术与应用探索[J]. 大数据, 2022, 8(2): 120-133.
[6]	谭玲, 鄂海红, 匡泽民, 宋美娜, 刘毓, 陈正宇, 谢晓璇, 李峻迪, 范家伟, 王晴川, 康霄阳. 医学知识图谱构建关键技术及研究进展[J]. 大数据, 2021, 7(4): 80-104.
[7]	刘庆霞, 李俊宥, 程龚. 实体摘要系统的解释性评测[J]. 大数据, 2021, 7(3): 15-29.
[8]	申宇铭, 杜剑峰. 时态知识图谱补全的方法及其进展[J]. 大数据, 2021, 7(3): 30-41.
[9]	杜会芳, 王昊奋, 史英慧, 王萌. 知识图谱多跳问答推理研究进展、挑战与展望[J]. 大数据, 2021, 7(3): 60-79.
[10]	胡志磊, 靳小龙, 陈剑赟, 黄冠利. 事件图谱的构建、推理与应用[J]. 大数据, 2021, 7(3): 80-96.
[11]	陈强, 代仕娅. 基于金融知识图谱的会计欺诈风险识别方法[J]. 大数据, 2021, 7(3): 116-129.
[12]	谢冰, 彭鑫, 尹刚, 李宣东, 魏峻, 孙海龙. 基于大数据的软件智能化开发方法与环境[J]. 大数据, 2021, 7(1): 3-21.
[13]	邹艳珍, 王敏, 谢冰, 林泽琦. 基于大数据的软件项目知识图谱构造及问答方法[J]. 大数据, 2021, 7(1): 22-36.
[14]	陈伟, 叶宏杰, 周家宏, 魏峻. 基于领域知识的Docker镜像自动构建方法[J]. 大数据, 2021, 7(1): 64-75.
[15]	张建, 孟祥鑫, 孙海龙, 王旭, 刘旭东. 数据驱动的软件开发者智能协作技术[J]. 大数据, 2021, 7(1): 76-93.