跨空间域数据管理分布式共识算法：现状、挑战和展望

doi:10.11959/j.issn.2096-0271.2023040

大数据 ›› 2023, Vol. 9 ›› Issue (4): 3-15.doi: 10.11959/j.issn.2096-0271.2023040

• 专题：跨域数据管理 • 上一篇

跨空间域数据管理分布式共识算法：现状、挑战和展望

李伟明¹, 李彤¹^,², 张大方¹, 戴隆超¹^,², 柴云鹏¹^,²

¹ 中国人民大学信息学院，北京 100872
² 数据工程与知识工程教育部重点实验室，北京 100872

出版日期:2023-07-01 发布日期:2023-07-01
作者简介:李伟明（1999- ），男，中国人民大学信息学院硕士生，主要研究方向为分布式共识协议
李彤（1989- ），男，博士，中国人民大学信息学院副教授，主要研究方向为新一代互联网体系结构、跨域数据管理和大数据
张大方（1998- ），男，中国人民大学信息学院硕士生，主要研究方向为分布式共识协议
戴隆超（1996- ）男，中国人民大学信息学院硕士生，主要研究方向为跨域数据管理和大数据
柴云鹏（1983- ），男，博士，中国人民大学信息学院教授、博士生导师，中国人民大学理工学科建设处副处长、计算机科学与技术系主任，主要研究方向为数据库管理系统、存储系统、云计算
基金资助:
国家自然科学基金资助项目(61972402);国家自然科学基金资助项目(61972275);国家自然科学基金资助项目(62202473);中国人民大学建设世界一流大学（学科）基金资助项目

Distributed consensus algorithms for crossdomain data management: state-of-the-art, challenges and perspectives

Weiming LI¹, Tong LI¹^,², Dafang ZHANG¹, Longchao DAI¹^,², Yunpeng CHAI¹^,²

¹ School of Information, Renmin University of China, Beijing 100872, China
² Key Laboratory of Data Engineering and Knowledge Engineering, Beijing 100872, China

Online:2023-07-01 Published:2023-07-01
Supported by:
The National Natural Science Foundation of China(61972402);The National Natural Science Foundation of China(61972275);The National Natural Science Foundation of China(62202473);Fund for Building World-Class Universities (Disciplines) of Renmin University of China

摘要/Abstract

摘要：

随着数字经济的飞速发展，以及“全国一体化数据中心”和“东数西算”等基础设施的不断完善，数据要素流通的大趋势使数据服务逐步由面向单一空间域的数据管理转变为面向跨空间域的数据管理。跨域数据管理需要通过分布式共识算法使数据一致。然而，已有的分布式共识算法仅考虑单数据中心的情况，没有考虑跨数据中心之间的网络通信的不确定性，从而在跨空间域场景下面临日志同步时延大、系统吞吐量低下等问题。系统地梳理了跨空间域下的分布式共识算法的现状以及面临的新挑战，并针对解决这些挑战的技术路线进行了展望。

关键词: 跨空间域数据管理, 分布式共识算法, 日志复制, 领导者选举

Abstract:

With the exponential growth of data and the company's cross-domain disaster recovery requirements, companies increasingly need to manage data across spatial domains.Cross-domain data management requires a distributed consensus algorithm to make the data consistent.However, the existing distributed consensus algorithms only consider the situation of a single data center, and do not consider the uncertainty of network communication between data centers, so they face long log synchronization delays and low system throughput in cross-space region scenarios and other issues.The current status and new challenges of distributed consensus algorithms in the cross-space domain were sorted out systematically, and the technical route to solve these challenges was looked forward.

Key words: cross-domain data management, distributed consensus algorithm, log replication, leader election

中图分类号:

TP319

李伟明, 李彤, 张大方, 戴隆超, 柴云鹏. 跨空间域数据管理分布式共识算法：现状、挑战和展望[J]. 大数据, 2023, 9(4): 3-15.

Weiming LI, Tong LI, Dafang ZHANG, Longchao DAI, Yunpeng CHAI. Distributed consensus algorithms for crossdomain data management: state-of-the-art, challenges and perspectives[J]. Big Data Research, 2023, 9(4): 3-15.

图/表 6

图1

图2

图3

图4

图5

表1

参考文献 20

[1]	柴云鹏, 李彤, 范举 ,等. 跨域数据管理的内涵与挑战[J]. 中国计算机学会通讯, 2022,18(11): 29-33.
	CHAI Y P , LI T , FAN J ,et al. Crossdomain data management:connotation and challenges[J]. Communications of China Computer Federation, 2022,18(11): 29-33.
[2]	ZAMANI M , MOVAHEDI M , RAYKOVA M . RapidChain:scaling blockchain via full sharding[C]// Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2018: 931-948.
[3]	AMIRI M J , LAI Z L , PATEL L ,et al. Saguaro:efficient processing of transactions in wide area networks using a hierarchical permissioned blockchain[J]. arXiv preprint, 2021,arXiv:2101.08819.
[4]	AMIRI M J , SHU D , MAIYYA S ,et al. Ziziphus:scalable data management across byzantine edge severs[C]// Proceedings of the 2023 IEEE 38th International Conference on Data Engineering. Piscataway:IEEE Press, 2023.
[5]	CHANDRA T D , GRIESEMER R , REDSTONE J . Paxos made live:an engineering perspective[C]// Proceedings of the 26th Annual ACM Symposium on Principles of Distributed Computing. New York:ACM Press, 2007: 398-407.
[6]	LAMPORT L . Paxos made simple[J]. ACM Sigact News, 2001,32(4): 18-25.
[7]	LAMPORT L . The part-time parliament[J]. ACM Transactions on Computer Systems, 1998,16(2): 133-169.
[8]	JUNQUEIRA F P , REED B C , SERAFINI M . Zab:high-performance broadcast for primary-backup systems[C]// Proceedings of 2011 IEEE/IFIP 41st International Conference on Dependable Systems ＆Networks (DSN). Piscataway:IEEE Press, 2011: 245-256.
[9]	ONGARO D , OUSTERHOUT J . In search of an understandable consensus algorithm[C]// Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference. New York:ACM Press, 2014: 305-320.
[10]	HUANG D X , LIU Q , CUI Q ,et al. TiDB:a raft-based HTAP database[J]. Proceedings of the VLDB Endowment, 2020,13(12): 3072-3084.
[11]	CAO W , ZHANG Y , YANG X ,et al. PolarDB serverless:a cloud native database for disaggregated data centers[C]// Proceedings of the 2021 International Conference on Management of Data. New York:ACM Press, 2021: 2477-2489.
[12]	TAFT R , SHARIF I , MATEI A ,et al. CockroachDB:the resilient geodistributed SQL database[C]// Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2020: 1493-1509.
[13]	JUN S M , KANG Y , KIM J ,et al. Ultralow-latency services in 5G systems:a perspective from 3GPP standards[J]. ETRI Journal, 2020,42(5): 721-733.
[14]	FINN N , THUBERT P , VARGA B ,et al. Deterministic networking architecture[J]. RFC, 2019,8655: 1-38.
[15]	SONG F , LI L T , YOU I ,et al. Enabling heterogeneous deterministic networks with smart collaborative theory[J]. IEEE Network, 2021,35(3): 64-71.
[16]	郑秀丽, 蒋胜, 王闯 . NewIP:开拓未来数据网络的新连接和新能力[J]. 电信科学, 2019,35(9): 2-11.
	ZHENG X L , JIANG S , WANG C . NewIP:new connectivity and capabilities of upgrading future data network[J]. Telecommunications Science, 2019,35(9): 2-11.
[17]	PARK S J , OUSTERHOUT J . Exploiting commutativity for practical fast replication[J]. arXiv preprint, 2017,arXiv:1710.09921.
[18]	MORARU I , ANDERSEN D G , KAMINSKY M . There is more consensus in Egalitarian parliaments[C]// Proceedings of the 24th ACM Symposium on Operating Systems Principles. New York:ACM Press, 2013: 358-372.
[19]	XU J J , WANG W , ZENG Y ,et al. Raft-PLUS:improving raft by multipolicy based leader election with unprejudiced sorting[J]. Symmetry, 2022,14(6): 1122.
[20]	NAWAB F , AGRAWAL D , EL ABBADI A . DPaxos:managing data closer to users for low-latency and mobile applications[C]// Proceedings of the 2018 International Conference on Management of Data. New York:ACM Press, 2018: 1221-1236.

优化方向	解决方案	优点	缺点
日志复制	尽早发送数据	缩短跨空间域请求的耗时	需要实时监控集群状态，选择合适的心跳时间
	分时发送数据	减小领导者在网络阻塞时期发送数据的压力	对网络状况预测准确度要求高
领导者选举	选举最优领导者	能够通过网络状态实时选取性能最优的领导者节点	视算法复杂度不同，系统的选举过程将产生额外的开销
	领导者主动禅让	当领导者节点网络状况发生变化时，能够及时切换到当前最优的领导者节点

跨空间域数据管理分布式共识算法：现状、挑战和展望

Distributed consensus algorithms for crossdomain data management: state-of-the-art, challenges and perspectives

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 20

相关文章 5

Metrics

推荐阅读 0

[1]	霍建同, 肖利民, 霍志胜, 徐耀文. 广域虚拟数据空间中边缘缓存系统的研究与实现[J]. 大数据, 2021, 7(5): 57-81.
[2]	沈逸, 姚旭, 朱扬勇. 数据自治开放与治理模式创新[J]. 大数据, 2018, 4(2): 14-20.
[3]	陈德华, 潘乔, 王梅, 乐嘉锦. 数据自治开放应用平台设计与实践[J]. 大数据, 2018, 4(2): 63-71.
[4]	蒋凡. 物流大数据标准及案例研究[J]. 大数据, 2017, 3(4): 60-66.
[5]	陈纯. 流式大数据实时处理技术、平台及应用[J]. 大数据, 2017, 3(4): 1-8.