满足本地化计算的集群资源调度策略

doi:10.3969/j.issn.1000-436x.2014.z2.001

通信学报 ›› 2014, Vol. 35 ›› Issue (Z2): 1-8.doi: 10.3969/j.issn.1000-436x.2014.z2.001

• 学术论文 • 下一篇

满足本地化计算的集群资源调度策略

郭平^1,²,宁立江¹,陈海珠³

¹ 重庆大学计算机学院，重庆 400044
² 软件理论与技术重庆市重点实验室，重庆 400044
³ 重庆电子工程职业学院，重庆 401331

出版日期:2014-11-25 发布日期:2017-06-19
基金资助:
国家自然科学青年基金资助项目;重庆市自然基金资助项目;重庆市自然基金资助项目;重庆市自然基金资助项目

Scheduling strategy for achieving locality in cluster

Ping GUO^1,²,Li-jiang NING¹,Hai-zhu CHEN³

¹ College of Computer Science,Chongqing University,Chongqing 400044,China
² Chongqing Key Laboratory of Software Theory ＆ Technology,Chongqing 400044,China
³ Department of Software Engineering,Chongqing College of Electronic Engineering,Chongqing 401331,China

Online:2014-11-25 Published:2017-06-19
Supported by:
The National Science Foundation for Young Scholars of China;The Natural Science Foundation Project of Chongqing CSTC;The Natural Science Foundation Project of Chongqing CSTC;The Natural Science Foundation Project of Chongqing CSTC

摘要/Abstract

摘要：

将本地化计算分成节点本地化计算和机架本地化计算2个层次，分别称作节点本地化计算和机架本地化计算。结合主导资源公平调度策略 DRF 和 Delay 调度约束机制提出一种满足本地化计算的集群资源调度策略DDRF，旨在既能达到一个较高的本地化计算水平，又能保证资源调度的“公平性”。通过数据实验分析了在DDRF调度策略下本地化计算延迟对作业执行效率的影响。

关键词: DDRF, 资源调度, 本地化计算, 计算集群

Abstract:

The data locality is divided into two levels.One is called the node data locality,which placing tasks on nodes that contain their input data.The other one is called the rack data locality,which placing tasks on nodes whose rack contains their input data.A new scheduling strategy called DDRF is proposed which combines the DRF and the delay.The DDRF is not only able to meet high locality but also achieve fairness.In the DDRF,the simulation results show the influence on the efficiency of jobs’ implement.

Key words: DDRF, scheduling, locality, computing cluster

郭平,宁立江,陈海珠. 满足本地化计算的集群资源调度策略[J]. 通信学报, 2014, 35(Z2): 1-8.

Ping GUO,Li-jiang NING,Hai-zhu CHEN. Scheduling strategy for achieving locality in cluster[J]. Journal on Communications, 2014, 35(Z2): 1-8.

图/表 6

表1

表2

图1

图2

图3

图4

参考文献 14

[1]	FIFO[EB/OL]. .
[2]	LEBOUDEC J Y . Rate adaptation,congestion control and fairnes s:a tutorial[EB/OL]. .
[3]	Capacity scheduler[EB/OL]. .
[4]	ZAHARIA M , BORTHAKUR D , SARMA J S ,et al. Job Scheduling for Multi-user Mapreduce Clusters:UCB/EECS-2009-55[R]. Berkeley:EECS Department,University of California, 2009.
[5]	GHODSI A , ZAHARIA M , HINDMAN B ,et al. Dominant resource fairness:fair allocation of multiple resource types[A]. Proceedings of the 8th USENIX Conference on NSDI[C]. Berbeley:USENIX Association, 2011. 24-37.
[6]	ZAHARIA M , KONWINSKI A , JOSEPH A D ,et al. Improving mapreduce performance in heterogeneous environments[A]. 8th USENIX Symposium on Operating Systems Design and Implementation[C]. San Diego:USENIX Association, 2008. 29-42.
[7]	POLO J , CARRERA D , BECERRA Y ,et al. Performance-driven task co-scheduling for mapreduce environments[A]. Network Operations and Management Symposium (NOMS)[C]. San Diego:IEEE, 2010. 373-380.
[8]	KC K , ANYANWU K . Scheduling hadoop jobs to meet deadlines[A]. Cloud Computing Technology and Science (CloudCom)[C]. San Diego:IEEE, 2010: 388-392.
[9]	Dynamic priority scheduling[EB/OL]. .
[10]	ZAHARIA M , BORTHAKUR D , SEN SARMA J ,et al. Delay scheduling:a simple technique for achieving locality and fairness in cluster scheduling[A]. Proceedings of the 5th European conference on Computer systems[. Paris:ACM, 2010: 265-278.
[11]	SHVACHKO K , KUANG H , RADIA S ,et al. The hadoop distributed file system[A]. Mass Storage Systems and Technologies (MSST)[C]. Lake Tahoe,Nevada,USA:IEEE, 2010: 1-10.
[12]	GUO P , NING L , SU L ,et al. A new strategy of resource management for cloud computing[J]. Information Technology Journal, 2013,12(18).
[13]	Amazon elastic compute cloud[EB/OL]. .
[14]	OSTERMANN S , IOSUP A , YIGITBASI N ,et al. A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing Cloud Computing[M]. Springer Berlin Heidelberg, 2010. 115-131.

调度序列	A		B		CPU使用量	内存使用量
调度序列	资源占比	主导资源占比	资源占比	主导资源占比	CPU使用量	内存使用量
B	＜0,0＞	0	＜3/9,1/18＞	1/3	3/9	1/18
A	＜1/9,4/18＞	2/9	＜3/9,1/18＞	1/3	4/9	5/18
A	＜2/9,8/18＞	4/9	＜3/9,1/18＞	1/3	5/9	9/18
B	＜2/9,8/18＞	4/9	＜6/9,2/18＞	2/3	8/9	10/18
A	＜3/9,12/18＞	2/3	＜6/9,2/18＞	2/3	1	14/18

E₁	p
10	40%
20	65%
30	79%
40	87%
50	93%
60	98%

满足本地化计算的集群资源调度策略

Scheduling strategy for achieving locality in cluster

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 14

相关文章 12

Metrics

推荐阅读 0

[1]	王莉, 费爱国, 张平, 徐连明. 智能应急指挥通信网络新框架与关键技术研究[J]. 通信学报, 2023, 44(6): 1-11.
[2]	赵庶旭, 韦萍, 王小龙. 多任务并发边缘计算环境中最优联盟结构生成策略[J]. 通信学报, 2023, 44(2): 172-184.
[3]	何元智, 彭聪, 于季弘, 刘韵. 面向密集多波束组网的卫星通信系统资源调度算法[J]. 通信学报, 2021, 42(4): 109-118.
[4]	李梓杨,于炯,王跃飞,卞琛,蒲勇霖,张译天,刘宇. Flink环境下基于负载预测的弹性资源调度策略[J]. 通信学报, 2020, 41(10): 92-108.
[5]	李梓杨,于炯,卞琛,张译天,蒲勇霖,王跃飞,鲁亮. 基于流网络的Flink平台弹性资源调度策略[J]. 通信学报, 2019, 40(8): 85-101.
[6]	苏命峰,王国军,李仁发. 基于利益相关视角的多维QoS云资源调度方法[J]. 通信学报, 2019, 40(6): 102-115.
[7]	王睿,韩笑冬,王超,周晞,龙军. 天基信息网络资源调度与协同管理[J]. 通信学报, 2017, 38(Z1): 104-109.
[8]	郭平，宁立江，陈海珠. 满足本地化计算的集群资源调度策略[J]. 通信学报, 2014, 35(Z2): 1-8.
[9]	夏纯中1,2，宋顺林1. 基于商空间的层次式数据网格资源调度算法[J]. 通信学报, 2013, 34(6): 18-155.
[10]	夏纯中,宋顺林. 基于商空间的层次式数据网格资源调度算法[J]. 通信学报, 2013, 34(6): 146-155.
[11]	覃光成,尹浩,陈强,吴泽民,杨盘隆. 面向价值的战场信息处理与分发优化算法[J]. 通信学报, 2011, 32(3): 60-68.
[12]	宋莉,胡立栓,肖沣,项彩虹. 基于OGSA的计算资源调度的一种实现[J]. 通信学报, 2005, 26(1A): 163-166.