基于RDMA和NVM的大数据系统一致性协议研究

doi:10.11959/j.issn.2096-0271.2019034

Abstract

Abstract:

Distributed storage systems and computing systems are the foundation for constructing big data processing systems.High availability of the system is the cornerstone of any distributed system.High-availability technologies generally rely on consensus protocols.The classic non-Byzantine distributed consensus protocol was discussed,as well as the RDMA communication protocol and NVM storage media under the development of new technologies to achieve higher performance high availability systems by combining them.The consensus protocol to make the better use of the features of RDMA and NVM was modified.The implemented system effectively improves the performance of the protocol while ensuring the consistency and availability of the system data.Experiments show that the system implemented in this paper can achieve 40% performance improvement compared to existing systems.

Key words: big data, distributed system, consensus protocol, RDMA, NVM

CLC Number:

TP302.8

Hao WU,Kang CHEN,Yongwei WU,Weimin ZHENG. Research on the consensus of big data systems based on RDMA and NVM[J]. Big Data Research, 2019, 5(4): 89-99.

Figures/Tables 7

References 32

[1]	GRAY J , LAMPORT L . Consensus on transaction commit[J]. ACM Transactions on Database Systems (TODS), 2006,31(1): 133-160.
[2]	SKEEN D , . Nonblocking commit protocols[C]// The 1981 ACM SIGMOD International Conference on Management of Data,April 29-May 1,1981,Ann Arbor,USA. New York:ACM Press, 1981: 133-142.
[3]	GIFFORD D K , . Weighted voting for replicated data[C]// The 7th ACM Symposium on Operating Systems Principles,December 10-12,1979,Pacific Grove,USA. New York:ACM Press, 1979: 150-162.
[4]	LAMPORT L . Paxos made simple[J]. ACM Sigact News, 2001,32(4): 18-25.
[5]	CHANDRA T D , GRIESEMER R , REDSTONE J . Paxos made live:an engineering perspective[C]// The 26th Annual ACM Symposium on Principles of Distributed Computing,August 12-15,2007,Portland,USA. New York:ACM Press, 2007: 398-407.
[6]	HUNT P , KONAR M , JUNQUEIRA F P ,et al. ZooKeeper:wait-free coordination for Internet-scale systems[C]// USENIX Annual Technical Conference,June 23-25,2010,Boston,USA.[S.l.:s.n]. 2010.
[7]	ONGARO D , OUSTERHOUT J . In search of an understandable consensus algorithm[C]// 2014 USENIX Annual Technical Conference,Junuary 17-20,2014,Philadelphia,USA.[S.l.:s.n. ], 2014: 305-319.
[8]	LAMPORT L . Fast Paxos[J]. Distributed Computing, 2006,19(2): 79-103.
[9]	ZHAO W . Fast Paxos made easy:theory and implementation[J]. International Journal of Distributed Systems and Technologies (IJDST), 2015,6(1): 15-33.
[10]	LAMPORT L B . Generalized Paxos:U.S.patent 7698465[P].2010-04-13.
[11]	SUTRA P , SHAPIRO M . Fast genuine generalized consensus[C]// 2011 IEEE 30th International Symposium on Reliable Distributed Systems,October 4-7,2011,Madrid,Spain. Piscataway:IEEE Press, 2011: 255-264.
[12]	MORARU I , ANDERSEN D G , KAMINSKY M . There is more consensus in egalitarian parliaments[C]// The 24th ACM Symposium on Operating Systems Principles,November 3-6,2013,Farminton,USA. New York:ACM Press, 2013: 358-372.
[13]	BURROWS M , . The chubby lock service for loosely-coupled distributed systems[C]// The 7th Symposium on Operating Systems Design and Implementation,November 6-8,2006,Seattle,USA. Berkeley:USENIX Association, 2006: 335-350.
[14]	BAKER J , BOND C , CORBETT J C ,et al. Megastore:providing scalable,highly available storage forinter active services[C]// The 5th Biennial CIDR Conference,January 9-11,2011,Asilomar,USA.[S.l.:s.n]. 2011: 223-234.
[15]	HUNT P , KONAR M , JUNQUEIRA F P ,et al. ZooKeeper:wait-free coordination for internet-scalesystems[C]// USENIX Annual Technical Conference,June 23-25,2010,Boston,USA. Berkeley:USENIX Association, 2010.
[16]	GUO C , WU H , DENG Z ,et al. RDMA over commodity ethernet at scale[C]// The 2016 ACM SIGCOMM Conference,August 22-26,2016,Florianopolis,Brazil. New York:ACM Press, 2016: 202-215.
[17]	TSAI S Y , ZHANG Y . Lite kernel RDMA support for datacenter applications[C]// The 26th Symposium on Operating Systems Principles,October 28-31,2017,Shanghai,China. New York:ACM Press, 2017: 306-324.
[18]	MACARTHUR P , RUSSELL R D . A performance study to guide RDMA programming decisions[C]// 2012 IEEE 14th International Conference on High Performance Computing and Communication,June 25-27,2012,Liverpool,UK. Piscataway:IEEE Press, 2012: 778-785.
[19]	CHEN A . A review of emerging nonvolatile memory (NVM) technologies and applications[J]. Solid-State Electronics, 2016(125): 25-38.
[20]	MAOW , LIU JN , TONG W ,et al. A review of storage technology research based on phase change memory[J]. Chinese Journal of Computers, 2015,38(5):944.
[21]	WANG C , JIANG J , CHEN X ,et al. Apus:fast and scalable Paxos on RDMA[C]// The 2017 Symposiumon Cloud Computing,September 25-27,2017,Santa Clara,USA. New York:ACM Press, 2017: 94-107.
[22]	KIM D , MEMARIPOUR A , BADAM A ,et al. Hyperloop:group-based NIC-offloading to accelerate replicated transactions in multi-tenant storage systems[C]// The 2018 Conference of the ACM Special Interest Group on Data Communication,August 20-25,2018,Budapest,Hungary. New York:ACM Press, 2018: 297-312.
[23]	DANG H T , SCIASCIA D , CANINI M ,et al. Netpaxos:consensus at network speed[C]// The 1st ACM SIGCOMM Symposium on Software Defined Networking Research,June 17-18,2015,Santa Clara,USA. New York:ACM Press, 2015.
[24]	MCKEOWN N , ANDERSON T , BALAKRISHNAN H ,et al. OpenFlow:enabling innovation in campus networks[J]. ACM SIGCOMM Computer Communication Review, 2008,38(2): 69-74.
[25]	LI J , MICHAEL E , SHARMA N K ,et al. Just say noto Paxos overhead:replacing consensus with network ordering[C]// The 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI16),November 2-4,2016,Savannah,USA. Berkeley:USENIX Association, 2016: 467-483.
[26]	BOSSHART P , DALY D , GIBB G ,et al. P4:programming protocol-independent packet processors[J]. ACM SIGCOMM Computer Communication Review, 2014,44(3): 87-95.
[27]	DANG H T , CANINI M , PEDONE F ,et al. Paxos made switch-y[J]. ACM SIGCOMM Computer Communication Review, 2016,46(2): 18-24.
[28]	SULAIMAN N , OBAID Z A , MARHABAN M ,et al. Design and implementation of FPGAbased systems-areview[J]. Australian Journal of Basic and Applied Sciences, 2009,3(4): 3575-3596.
[29]	ISTVáN Z , SIDLER D , ALONSO G ,et al. Consensus in a box:inexpensive coordination in hardware[C]// The 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI16),March 16-18,2016,Santa Clara,USA. Berkeley:USENIX Association, 2016: 425-438.
[30]	POKE M , HOEFLER T . DARE:highperformance state machine replication on RDMA networks[C]// The 24th International Symposium on High-Performance Parallel and Distributed Computing,June 15-19,2015,Portland,USA. New York:ACM Press, 2015: 107-118.
[31]	TALEB Y , STUTSMAN R , ANTONIU G ,et al. Tailwind:fast and atomic RDMA-based replication[C]// The 2018 USENIX Annual Technical Conference,July 11-13,2018,Boston,USA.[S.l.:s.n. ], 2018: 851-863.
[32]	PANDA D K , KOOP M , BALAJI P . Tutorial:infiniband and 10-Gigabit Ethernet for dummies[C]// IEEE Symposium on High Performance Interconnects,August 25-27,2009,New York,USA. Piscataway:IEEE Press, 2009.

Metrics

Recommended 0

No Suggested Reading articles found!

原语	UD	UC	RC
Send	√	√	√
Receive	√	√	√
RDMA Write		√	√
RDMA Read			√
Atomic			√

特性	磁盘	PCM	DRAM
读能耗/(J/GB)	65	1	0.8
写能耗/(J/GB)	65	6	1.2
读时延	~5 ms	~60 ns	20~50 ns
写时延	~5 ms	50~120 ns	20~50 ns
最大擦写次数/次	∞	10⁶~10⁸	∞

配置	具体信息
操作系统	CentOS 7.2
CPU	Intel Xeon CPU E5-2643 v4 12核，主频3.40 GHz
内存	128 GB
网卡	Mellnaox ConnectX-3 Infiniband网卡，56 Gbit/s

Research on the consensus of big data systems based on RDMA and NVM

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 32

Related Articles 15

Metrics

Recommended 0

[1]	Haihong QIAN, Maoyi WANG, Yun XIONG. Digital transformation in higher education:a systematic review [J]. Big Data Research, 2023, 9(3): 56-70.
[2]	Hong MEI, Xiaoyong DU, Hai JIN, Xueqi CHENG, Yunpeng CHAI, Xuanhua SHI, Xiaolong JIN, Yasha WANG, Chi LIU. Big data technologies forward-looking [J]. Big Data Research, 2023, 9(1): 1-20.
[3]	Yang SHEN, Menglong YU. Metaverse and big data: data insight and value connection in spatio-temporal intelligence [J]. Big Data Research, 2023, 9(1): 103-110.
[4]	Tongzheheng ZHENG, Bin LI, Minxuan FENG, Bolin CHANG, Dongbo WANG. Explore the structuration of historical books:the construction and quantitative analysis of digital humanities database of the Biographies of the Shiji [J]. Big Data Research, 2022, 8(6): 40-55.
[5]	Jing CHEN. Humanities big data and its application in the field of digital humanities [J]. Big Data Research, 2022, 8(6): 3-14.
[6]	Yuchu LUO, Hao WU, Yuhan GUO, Shaocong TAN, Can LIU, Ruike JIANG, Xiaoru YUAN. Visualization in digital humanities [J]. Big Data Research, 2022, 8(6): 74-93.
[7]	Wenlong LI, Yuan YUAN, Xiaopeng AN. Modus operandi of big data governance: some preliminary observations [J]. Big Data Research, 2022, 8(4): 34-45.
[8]	Qifeng TANG, Zhiqing SHAO, Yazhen YE. Authenticating and licensing architecture of data rights in data trade [J]. Big Data Research, 2022, 8(3): 40-53.
[9]	Chenhuizi WANG, Wei CAI. Digital economics in metaverse: state-of-the-art, characteristics, and vision [J]. Big Data Research, 2022, 8(3): 140-150.
[10]	Mei YANG, Wei LI, Siyuan QIAO, Wei LIU. Research on calculation method of China’s big data industry output value [J]. Big Data Research, 2022, 8(3): 151-160.
[11]	Deren LI, Guo ZHANG, Yonghua JIANG, Xin SHEN, Weiling LIU. Opportunities and challenges of geo-spatial information science from the perspective of big data [J]. Big Data Research, 2022, 8(2): 3-14.
[12]	Xiaolan QIU, Yuxin HU, Songtao SHANGGUAN, Kun FU. Remote sensing satellite big data high-recision integration processing technology [J]. Big Data Research, 2022, 8(2): 15-27.
[13]	Weiquan LIU, Cheng WANG, Yu ZANG, Qian HU, Shangshu YU, Baiqi LAI. A survey on information extraction technology based on remote sensing big data [J]. Big Data Research, 2022, 8(2): 28-57.
[14]	Jianqiang LIU, Xiaomin YE, Youguo LAN. Remote sensing big data from Chinese ocean satellites and its application service [J]. Big Data Research, 2022, 8(2): 75-88.
[15]	Hequn YANG, Xiaofeng WANG, Yanqing GAO, Yiwen LU, Bingxin MA, Xinyao WANG. Analysis of satellite big data requirements in numerical weather prediction [J]. Big Data Research, 2022, 8(2): 89-102.