群体知识图谱：分布式知识迁移与联邦式图谱推理

doi:10.11959/j.issn.2096-6652.202217

智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (1): 55-64.doi: 10.11959/j.issn.2096-6652.202217

所属专题：知识图谱

群体知识图谱：分布式知识迁移与联邦式图谱推理

陈名杨¹, 张文², 陈湘楠², 周虹廷¹, 陈华钧¹

¹ 浙江大学计算机科学与技术学院，浙江杭州 310007
² 浙江大学软件学院，浙江杭州 310007

修回日期:2022-02-25 出版日期:2022-03-15 发布日期:2022-03-01
作者简介:陈名杨（1997− ），男，浙江大学计算机科学与技术学院博士生，主要研究方向为知识图谱表示学习
张文（1992− ），女，浙江大学软件学院助理研究员，主要研究方向为知识图谱、大数据系统、图数据处理
陈湘楠（1998− ），男，浙江大学软件学院硕士生，主要研究方向为知识图谱表示学习
周虹廷（1997− ），女，浙江大学计算机科学与技术学院硕士生，主要研究方向为知识图谱和图神经网络
陈华钧（1978− ），男，博士，浙江大学计算机科学与技术学院教授，主要研究方向为知识图谱、大数据系统与自然语言处理
基金资助:
国家自然科学基金资助项目(91846204);国家自然科学基金资助项目(U19B2027)

Collective knowledge graph: meta knowledge transfer and federated graph reasoning

Mingyang CHEN¹, Wen ZHANG², Xiangnan CHEN², Hongting ZHOU¹, Huajun CHEN¹

¹ College of Computer Science and Technology, Zhejiang University, Hangzhou 310007, China
² School of Software Technology, Zhejiang University, Hangzhou 310007, China

Revised:2022-02-25 Online:2022-03-15 Published:2022-03-01
Supported by:
The National Natural Science Foundation of China(91846204);The National Natural Science Foundation of China(U19B2027)

摘要/Abstract

摘要：

群体知识图谱是指通过群体协作，以去中心化或分布式方式管理和维护的知识图谱。相比现有的集中式管理的知识图谱，群体知识图谱具备知识确权、隐私保护、众包激励、可信溯责等特点。尝试探讨构建或应用群体知识图谱平台面临的技术挑战。其中分布式知识迁移考虑在一个分散自治的框架下，通过实现不同来源的多个知识图谱之间的知识迁移，缓解单个知识图谱的知识不完备问题。其主要难点是在充分保护知识的自治所有权的前提下，尽可能共享有用的知识，以增强各自的知识图谱表示。联邦式图谱推理也是考虑在一个分布式环境下，通过联邦学习机制实现隐私保护前提下的知识图谱推理。在分布式知识迁移中，强调在关系集合互相重叠的知识图谱间迁移与实体无关的知识；而在联邦式图谱推理中，强调在多个实体集合互相重叠的知识图谱间共同学习更好的实体嵌入表示。针对这两个问题分别进行模型设计及实验验证。

关键词: 群体知识图谱, 联邦学习, 知识迁移

Abstract:

Collective knowledge graphs refer to knowledge graphs that are managed and maintained in a decentralized or distributed manner through group collaboration.Compared with the existing centrally managed knowledge graph, the collective knowledge graph has the characteristics of knowledge right confirmation, privacy protection, crowd sourcing incentive, and credible traceability.Tring to explore the technical challenges faced by building and applying a collective knowledge graph platform.For meta knowledge transfer, the knowledge incompleteness of a single knowledge graph by knowledge transfer among multiple knowledge graphs from different sources under a decentralized and autonomous framework was considered.The main difficulty was to enhance the respective knowledge graph representation by sharing useful knowledge with each other as much as possible while fully protecting the autonomous ownership of knowledge.For federated graph reasoning, the knowledge graph reasoning in a distributed environment under the privacy-preserving by means of the federated learning mechanism was considered.Meta knowledge transfer focused on transferring entity-independent knowledge between knowledge graphs with overlapped relation set, while federated graph reasoning aimed at learning better entity embeddings for knowledge graphs with overlapped entity set.The model design and experimental validation for each of these two problems were conducted.

Key words: collective knowledge graph, federated learning, knowledge transfer

中图分类号:

TP391

陈名杨, 张文, 陈湘楠, 等. 群体知识图谱：分布式知识迁移与联邦式图谱推理[J]. 智能科学与技术学报, 2022, 4(1): 55-64.

Mingyang CHEN, Wen ZHANG, Xiangnan CHEN, et al. Collective knowledge graph: meta knowledge transfer and federated graph reasoning[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 55-64.

图/表 5

图1

表1

不同知识图谱表示学习方法的打分函数"

知识图谱表示学习方法	打分函数	向量空间
TransE	$- ∥ h + r - t ∥$	$h, r, t \in ℝ^{d}$
DistMult	$h^{⊤} diag (r) t$	$h, r, t \in ℝ^{d}$
ComplEx	$Re (h^{⊤} diag (r) \bar{t})$	$h, r, t \in ℂ^{d}$
RotatE	$- ∥ h \circ r - t ∥$	$h, r, t \in ℂ^{d}$

表1

图2

表2

分布式知识迁移实验结果"

模型名称	WN18RR				FB15k-237				NELL-995
模型名称	v1	v2	v3	v4	v1	v2	v3	v4	v1	v2	v3	v4
Neural LP	74.4	69.0	46.2	67.1	52.9	58.9	52.9	55.9	40.8	78.7	82.7	$\underline{80.6}$
DRUM	74.4	69.0	46.2	67.1	52.9	58.7	52.9	55.9	19.4	78.6	82.7	$\underline{80.6}$
RuleN	80.9	78.2	53.4	71.6	49.8	77.8	$\underline{87.7}$	85.6	53.5	81.8	77.3	61.4
GraIL	82.5	78.7	58.4	73.4	64.1	81.8	82.8	$\underline{89.3}$	$\underline{59.5}$	93.3	91.4	73.1
CoMPILE	$\underline{83.6}$	$\underline{79.8}$	$\underline{60.7}$	$\underline{75.5}$	$\underline{67.6}$	$\underline{83.0}$	84.7	87.4	58.4	$\underline{93 . 9}$	$\underline{92 . 8}$	75.1
MorsE (TransE)	81.5	78.8	67.3	76.5	$84 . 7$	$96 . 3$	$95 . 8$	$96 . 4$	51.8	83.8	89.3	$82 . 4$
MorsE (DistMult)	83.7	80.0	66.1	79.3	82.5	94.9	94.9	95.8	$66 . 6$	79.4	84.3	54.1
MorsE (ComplEx)	83.5	81.4	67.3	79.4	80.0	92.8	93.6	95.1	65.0	76.9	82.3	42.6
MorsE (RotatE)	$84 . 1$	$81 . 5$	$70 . 9$	$79 . 6$	83.2	95.7	95.7	95.9	65.2	80.7	87.7	53.4

表2

表3

联邦式图谱推理实验结果"

知识图谱表示学习方法	设置	客户端1		客户端2		客户端3
知识图谱表示学习方法	设置	MRR	Hits@10	MRR	Hits@10	MRR	Hits@10
	isolation	50.4	69.8	38.7	63.7	33.7	59.7
TransE	collection	54.2	74.1	41.0	66.9	35.7	63.1
	FedE	$\underline{54 . 8}$	$\underline{74 . 8}$	$\underline{41 . 8}$	$\underline{68 . 4}$	$\underline{36 . 3}$	$\underline{63 . 4}$
	isolation	48.9	65.2	36.6	56.2	34.7	57.0
DistMult	collection	51.7	70.1	38.2	59.9	33.9	56.4
	FedE	$\underline{53 . 6}$	$\underline{70 . 3}$	$\underline{40 . 4}$	$\underline{62 . 4}$	$\underline{37 . 5}$	$\underline{61 . 5}$
	isolation	48.4	65.3	37.2	57.8	32.6	55.7
ComplEx	collection	50.0	67.5	36.9	57.4	33.4	55.5
	FedE	$\underline{52 . 9}$	$\underline{70 . 2}$	$\underline{40 . 9}$	$\underline{63 . 2}$	$\underline{36 . 1}$	$\underline{59 . 5}$
	isolation	52.7	70.6	40.6	64.2	36.6	60.2
RotatE	collection	$55 . 7$	75.1	42.1	67.3	37.1	63.5
	FedE	$\underline{55.5}$	$\underline{75 . 4}$	$\underline{43 . 0}$	$\underline{68 . 5}$	$\underline{38 . 9}$	$\underset{⇁}{65 . 0}$

表3

参考文献 27

[1]	BERNERS-LEE T , HENDLER J , LASSILA O . The semantic Web[J]. Scientific American, 2001,284(5): 34-43.
[2]	VRANDE?I? D KR?TZSCH M . Wikidata[J]. Communications of the ACM, 2014,57(10): 78-85.
[3]	AUER S , BIZER C , KOBILAROV G ,et al. DBpedia:a nucleus for a web of open data[M]. The semantic Web. Heidelberg: Springer, 2007.
[4]	MITCHELL T , COHEN W , HRUSCHKA E ,et al. Never-ending learning[J]. Communications of the ACM, 2018,61(5): 103-115.
[5]	XIONG W H , DU J F , WANG W Y ,et al. Pretrained encyclopedia:weakly supervised knowledge-pretrained language model[C]// Proceedings of the International Conference on Learning Representations.[S.l.:s.n.], 2019.
[6]	YASUNAGA M , REN H Y , BOSSELUT A ,et al. QA-GNN:reasoning with language models and knowledge graphs for question answering[C]// Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg:Association for Computational Linguistics, 2021: 535-546.
[7]	ZHANG Q , ZHANG L , QIN C ,et al. A survey on knowledge graph-based recommender systems[J]. Scientia Sinica Informationis, 2020,50(7): 937-956.
[8]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint, 2018,arXiv:1810.04805.
[9]	WANG X Z , GAO T Y , ZHU Z C ,et al. KEPLER:a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021,9: 176-194.
[10]	CHEN H J , HU N , QI G L ,et al. OpenKG chain:a blockchain infrastructure for open knowledge graphs[J]. Data Intelligence, 2021: 1-18.
[11]	BORDES A , USUNIER N , GARCIA-DURAN A , ,et al. Translating embeddings for modeling multi-relational data[J]. Advances in Neural Information Processing Systems, 2013,26.
[12]	WANG Z , ZHANG J W , FENG J L ,et al. Knowledge graph embedding by translating on hyperplanes[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2014.
[13]	ZHOU X H , YI Y H , JIA G . Path-RotatE:knowledge graph embedding by relational rotation of path in complex space[C]// Proceedings of 2021 IEEE/CIC International Conference on Communications in China. Piscataway:IEEE Press, 2021: 905-910.
[14]	NICKEL M , TRESP V , KRIEGEL H . A three-way model for collective learning on multi-relational data[C]// Proceedings of the 28th International Conference on International Conference on Machine Learning.[S.l.:s.n.], 2011: 809-816.
[15]	YANG B S , YIH W T , HE X D ,et al. Embedding entities and relations for learning and inference in knowledge bases[J]. arXiv preprint, 2014,arXiv:1412.6575.
[16]	TROUILLON T , WELBL J , Riedel S R ,et al. Complex embeddings for simple link prediction[C]// Proceedings of the International Conference on Machine Learning.[S.l.:s.n.], 2016: 2071-2080.
[17]	TERU K , DENIS E , HAMILTON W . Inductive relation prediction by subgraph reasoning[C]// Proceedings of the International Conference on Machine Learning.[S.l.:s.n.], 2020: 9448-9457.
[18]	MAI S J , ZHENG S J , YANG Y D ,et al. Communicative message passing for inductive relation reasoning[J]. arXiv preprint, 2020,arXiv:2012.08911.
[19]	CHEN J J , HE H R , WU F ,et al. Topology-aware correlations between relations for inductive link prediction in knowledge graphs[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2021: 6271-6278.
[20]	GALáRRAGA L A , TEFLIOUDI C , HOSE K ,et al. AMIE:association rule mining under incomplete evidence in ontological knowledge bases[C]// Proceedings of the 22nd International Conference on World Wide Web.[S.l.:s.n.], 2013: 413-422.
[21]	MEILICKE C , FINK M , WANG Y J ,et al. Fine-grained evaluation of rule- and embedding-based systems for knowledge graph completion[C]// Proceedings of the International Semantic Web Conference.[S.l.:s.n.], 2018: 3-20.
[22]	YANG F , YANG Z L , COHEN W W . Differentiable learning of logical rules for knowledge base reasoning[J]. arXiv preprint, 2017,arXiv:1702.08367.
[23]	SADEGHIAN A , ARMANDPOUR M , DING P ,et al. DRUM:end-to-end differentiable rule mining on knowledge graphs[J]. Advances in Neural Information Processing Systems, 2019,32: 15347-15357.
[24]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[J]. arXiv preprint, 2016,arXiv:1602.05629.
[25]	DETTMERS T , MINERVINI P , STENETORP P ,et al. Convolutional 2D knowledge graph embeddings[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2018.
[26]	TOUTANOVA K , CHEN D Q , PANTEL P ,et al. Representing text for joint embedding of text and knowledge bases[C]// Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2015: 1499-1509.
[27]	XIONG W H , HOANG T , WANG W Y . DeepPath:a reinforcement learning method for knowledge graph reasoning[C]// Proceedings of 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2017: 564-573.

群体知识图谱：分布式知识迁移与联邦式图谱推理

Collective knowledge graph: meta knowledge transfer and federated graph reasoning

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 27

相关文章 2

Metrics

推荐阅读 0

[1]	杨强, 童咏昕, 王晏晟, 范力欣, 王薇, 陈雷, 王魏, 康焱. 群体智能中的联邦学习算法综述[J]. 智能科学与技术学报, 2022, 4(1): 29-44.
[2]	魏雅婷,王智勇,周舒悦,陈为. 联邦可视化：一种隐私保护的可视化新模型[J]. 智能科学与技术学报, 2019, 1(4): 415-420.