云环境下大规模分布式计算数据感知的调度系统

doi:10.11959/j.issn.2096-0271.2020007

摘要/Abstract

摘要：

介绍了新的调度系统，包括资源调度、应用编排、配置标签中心、云网络和云存储服务等子系统。系统通过数据拓扑感知能力保证了计算和数据的局部性，节约网络I/O开销；通过优化点对点大数据量读取的资源调度，解决网络风暴造成的影响；通过网络和磁盘隔离技术以及可抢占的方式来保证服务等级协议。

关键词: 云计算, 调度系统, 大数据, AI平台, 数据局部性, 分布式计算, 抢占

Abstract:

A novel scheduler system including resource scheduling,application scheduling,configuration and label management center,cloud network and cloud storage services was introduced.The locality of computation and data was ensured by the ability of data topology awareness,and the I/O cost was saved.The impact of network storm was solved by optimizing the resource scheduling of point to point large data reading.The service level protocol was guaranteed by network and disk isolation technology and preemptive way.

Key words: cloud computing, scheduling system, big data, artificial intelligence platform, data locality, distributed computing, preemption

中图分类号:

TP315

刘汪根, 郑淮城, 荣国平. 云环境下大规模分布式计算数据感知的调度系统[J]. 大数据, 2020, 6(1): 81-98.

Wanggen LIU, Huaicheng ZHENG, Guoping RONG. A scheduler system for large-scale distributed data computing in cloud[J]. Big Data Research, 2020, 6(1): 81-98.

图/表 13

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

表1

图12

参考文献 18

[1]	周涛, 潘柱廷, 程学旗 . CCF大专委2019年大数据发展趋势预测[J]. 大数据, 2019,5(1): 109-115.
	ZHOU T , PAN Z T , CHENG X Q . Developing tendency prediction of big data in 2019 from CCF TFBD[J]. Big Data Research, 2019,5(1): 109-115.
[2]	BURNS B , GRANT B , OPPENHEIMER D ,et al. Borg,omega,and Kubernetes[J]. Communications of the ACM, 2016,59(5): 50-57.
[3]	ZAHARIA M , CHOWDHURY M , DAS T ,et al. Resilient distributed datasets:a fault-tolerant abstraction for in-memory cluster computing[C]// The 9th Usenix Conference on Networked Systems Design and Implementation,April 25-27,2012,San Jose,USA. Berkeley:USENIX Association, 2012.
[4]	BREWER E A , . Kubernetes and the path to cloud native[C]// The 6th ACM Symposium on Cloud Computing,August 27-29,2015,Kohala Coast,USA. New York:ACM Press, 2015:167.
[5]	VERMA A , PEDROSA L , KORUPOLU M ,et al. Large-scale cluster management at Google with Borg[C]// The 10th European Conference on Computer Systems,April 21-24,2015,Bordeaux,France. New York:ACM Press, 2015.
[6]	HINDMAN B , KONWINSKI A , ZAHARIA M ,et al. Mesos:a platform for finegrained resource sharing in the data center[C]// The 8th USENIX Conference on Networked Systems Design and Implementation,March 30-April 1,2011,Boston,USA. Berkeley:USENIX Association, 2013: 295-308.
[7]	SCHWARZKOPF M , KONWINSKI A , ABDEL-MALEK M . Omega:flexible,scalable schedulers for large compute clusters[C]// The 8th ACM European Conference on Computer Systems,April 15 - 17,2013,Prague,Czech Republic. New York:ACM Press, 2013: 351-364.
[8]	GHEMAWAT S , GOBIOFF H , LEUNG S T . The Google file system[J]. ACM SIGOPS Operating Systems Principles, 2003,37(5): 29-43.
[9]	CHANG F , DEAN J , GHEMAWAT S ,et al. Bigtable:a distributed storage system for structured data[J]. ACM Transactions on Computer System, 2008,26(2): 1-26.
[10]	DEAN J , GHEMAWAT S . MapReduce:simplified data processing on large clusters[C]// The 6th Conference on Symposium on Operating Systems Design＆ Implementation,December 6-8,2004,San Francisco,USA. Berkeley:USENIX Association, 2004:10.
[11]	LESLIE L . The part-time parliament[J]. ACM Transactions on Computer Systems, 1998: 133-169.
[12]	YANG F , TSCHETTER E , MERLINO G ,et al. Druid:a real-time analytical data store[C]// The 2014 ACM SIGMOD International Conference on Management of Data,June 2227,2014,Snowbird,USA. New York:ACM Press, 2014: 157-168.
[13]	PARIS C , STEPHAN E , SEIF H . Apache Flink:stream and batch processing in a single engine[J]. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015,36(4): 28-38.
[14]	ZAHARIA M , DAS T , LI H Y ,et al. Discretized streams:fault-tolerant streaming computation at scale[C]// The 24th ACM SOSP Symposium on Operating Systems Principles,November 3-6,2013,Farmington,USA. New York:ACM Press, 2013: 423-428.
[15]	MARTIN A , PAUL B , CHEN J M ,et al. TensorFlow:a system for large-scale machine learning[C]// The 12th USENIX Symposium on Operating Systems Design and Implementation,November 2-4,2016,Savannah,USA. Berkeley:USENIX Association, 2016: 265-283.
[16]	GANDINI A , GRIBAUDO M , KNOTTENBELT W J ,et al. Performance evaluation of NoSQL databases[M]. Heidelberg: SpringerPress, 2014.
[17]	NAMBIAR R O , POESS M . The making of TPC-DS[C]// The 32nd International Conference on Very Large Data Bases,September 12-15,2006,Seoul,Korea.[S.l.:s.n. ], 2006: 1049-1058.
[18]	ZAHARIA M , BORTHAKUR D , SARMA J S ,et al. Delay scheduling:a simple technique for achieving locality and fairness in cluster scheduling[C]// The 5th European Conference on Computer Systems,April 13-16,2010,Paris,France. New York:ACM Press, 2010: 265-278.

测试比对项	期望的结果与原因分析
租户1与主机部署	主机部署模式不存在随意的调度，所有的角色都是预先分配好的。租户1在云平台上调度，但是通过标签配置来干预其调度方式，限制其在10个节点上有限调度。如果租户1能够正常运行性能测试程序，并且性能接近主机模式，那么就可以证明基于标签的调度能力是有效的；如果性能相近，则证明调度系统对于计算和数据局部性、网络感知能力也是有效的
租户1与租户2	租户2和租户1相比，能够在更多的机器上调度，因此相对来说，计算和数据更为离散，并且可能会存在网络瓶颈。可以通过对比测试来验证随着集群规模扩大，调度系统是否可以保证稳定的调度能力，其存储和网络感知能力是否在调度系统中有效发挥作用
租户3与租户2	租户3和租户2同样都是在大规模的节点上调度的分布式系统，租户3使用的Kubernetes调度系统不会感知应用和底层数据、网络、计算等因素，因此预期租户3的测试性能和稳定性都会差一些
租户2与主机部署	期望租户2在真实云环境下运行大数据或者AI应用，即在一个大的云平台上运行海量的应用，用户无需感知细节；而主机部署方式也就是现在主流使用的采用裸金属机器部署大数据和AI的方式，没有弹性，并且不能保证资源使用率；如果租户2和主机部署方式的性能比较相近，那就表明能够在不牺牲性能的前提下，实现弹性能力以及更高的资源使用率