基于强化学习的在线离线混部云环境下的调度框架

doi:10.11959/j.issn.1000-436x.2023119

Abstract

Abstract:

Some reinforcement learning-based scheduling algorithms for cloud computing platforms barely considered one scenario or ignored the resource constraints of jobs and treated all machines as the same type, which caused low resource utilization or insufficient scheduling efficiency.To address the scheduling problems in online-offline colocated cloud environment, a framework named JobFusion was proposed.Firstly, an efficient resource partitioning scheme was built in the cloud computing platform supporting virtualization technology by integrating the hierarchical clustering method with connectivity constraints.Secondly, a graph convolutional neural network was utilized to embed the attributes of elastic dimension with various constraints and the jobs with various numbers, to capture the critical path information of workflow.Finally, existing high-performance reinforcement learning methods were integrated for scheduling jobs.According to the results of evaluation experiments, JobFusion improves the resource utilization by 39.86% and reduces the average job completion time by up to 64.36% compared with baselines.

Key words: reinforcement learning, graph embedding, hierarchical cluster, cloud computing, virtualization

CLC Number:

TP18

Ling MA, Qiliang FAN, Ting XU, Guanchen GUO, Shenglin ZHANG, Yongqian SUN, Yuzhi ZHANG. Scheduling framework based on reinforcement learning in online-offline colocated cloud environment[J]. Journal on Communications, 2023, 44(6): 90-102.

Figures/Tables 17

References 15

[1]	SRIVASTAVA P , KHAN R . A review paper on cloud computing[J]. International Journal of Advanced Research in Computer Science and Software Engineering, 2018,8(6): 17.
[2]	MAO H Z , SCHWARZKOPF M , VENKATAKRISHNAN S B ,et al. Learning scheduling algorithms for data processing clusters[C]// Proceedings of the ACM Special Interest Group on Data Communication. New York:ACM Press, 2019: 270-288.
[3]	MNIH V , BADIA A P , MIRZA M ,et al. Asynchronous methods for deep reinforcement learning[C]// Proceedings of the 33rd International Conference on International Conference on Machine Learnin. New York:ACM Press, 2016: 1928-1937.
[4]	YU H F , IRISSAPPANE A A , WANG H ,et al. FaaSRank:learning to schedule functions in serverless platforms[C]// Proceedings of 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS). Piscataway:IEEE Press, 2022: 31-40.
[5]	ZHANG Y , ZHU H , TANG D ,et al. Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems[J]. Robotics and Computer-Integrated Manufacturing, 2022:doi.org/10.1016/j.rcim.2022.102412.
[6]	SONG W , CHEN X Y , LI Q Q ,et al. Flexible job-shop scheduling via graph neural network and deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics, 2023,19(2): 1600-1610.
[7]	ZHOU G , WEN R , TIAN W ,et al. Deep reinforcement learning-based algorithms selectors for the resource scheduling in hierarchical cloud computing[J]. Journal of Network and Computer Applications, 2022,208: 103520
[8]	SHENG J , HU Y , ZHOU W ,et al. Learning to schedule multi-NUMA virtual machines via reinforcement learning[J]. Pattern Recognition, 2022,121: 108254.
[9]	LI W G , LI J L , WU W G . A dynamic scheduling algorithm with time varying resource constraints in colocation data centers[C]// Proceedings of the 13th International Conference on Information and Communication Systems (ICICS). Piscataway:IEEE Press, 2022: 115-120.
[10]	ZHANG X L , LI L Q , WANG Y ,et al. Zeus:improving resource efficiency via workload colocation for massive kubernetes clusters[J]. IEEE Access, 2021,9: 105192-105204.
[11]	PI A D , ZHOU X B , XU C Z . Holmes:SMT interference diagnosis and CPU scheduling for job co-location[C]// Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing. New York:ACM Press, 2022: 110-121.
[12]	SUNNY M , SEN S J , SABU S E ,et al. Deploy-Web hosting using docker container[C]// Advances in Computing and Network Communications. Singapore:Springer, 2021: 335-345.
[13]	JAYAPRAKASH S , NAGARAJAN M D , PRADO R P D ,et al. A systematic review of energy management strategies for resource allocation in the cloud:clustering,optimization and machine learning[J]. Energies, 2021:doi.org/10.3390/en14175322.
[14]	CHHAJED D , LOWE T J . Building intuition:insights from basic operations management models and principles[M]. New York: Springer, 2008.
[15]	XU H W , CHEN W X , ZHAO N W ,et al. Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications[C]// Proceedings of the 2018 World Wide Web Conference. New York:ACM Press, 2018: 187-196.

Metrics

Recommended 0

No Suggested Reading articles found!

硬件	规格参数
CPU	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20 GHz×2
DRAM	128 GB

物理机资源类型	CPU资源/核	内存资源/GB	机器数量/台
类型1	32	64	6 000
类型2	92	288	3 000

Scheduling framework based on reinforcement learning in online-offline colocated cloud environment

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 15

Related Articles 15

Metrics

Recommended 0

[1]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[2]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[3]	Dacheng ZHOU, Hongchang CHEN, Weizhen HE, Guozhen CHENG, Hongchao HU. Research on multidimensional dynamic defense strategy for microservice based on deep reinforcement learning [J]. Journal on Communications, 2023, 44(4): 50-63.
[4]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[5]	Wenjun XU, Silei WU, Fengyu WANG, Lan LIN, Guojun LI, Zhi ZHANG. Large-scale post-disaster user distributed coverage optimization based on multi-agent reinforcement learning [J]. Journal on Communications, 2022, 43(8): 1-16.
[6]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[7]	Zexi XU, Lei ZHUANG, Kunli ZHANG, Mingyu GUI. Online placement algorithm of service function chain based on knowledge graph [J]. Journal on Communications, 2022, 43(8): 41-51.
[8]	Shuai MA, Bing LI, Haihong SHENG, Rongyan GU, Hui ZHOU, Hongmei WANG, Yue WANG, Shiyin LI. Research on power allocation of integrated VLPC based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 121-130.
[9]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[10]	Yanfei SUN, Jiazheng YIN, Jin QI, Xiaoxuan HU, Mengting CHEN, Zhenjiang DONG. Topology control based on dynamic graph embedding in Internet of vehicles [J]. Journal on Communications, 2022, 43(6): 133-142.
[11]	Peiliang ZUO, Shaolong HOU, Chao GUO, Hua JIANG, Wenbo WANG. Security decision method for the edge of multi-layer satellite network based on reinforcement learning [J]. Journal on Communications, 2022, 43(6): 189-199.
[12]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[13]	Chuanhuang LI, Yangting CHEN, Jingjing TANG, Jiali LOU, Renhua XIE, Chuntao FANG, Weiming WANG, Chao CHEN. QL-STCT: an intelligent routing convergence method for SDN link failure [J]. Journal on Communications, 2022, 43(2): 131-142.
[14]	Hang QIU, Hongbo TANG, Wei YOU, Yu ZHAO, Yi BAI. QGA-based network service extension algorithm in NFV [J]. Journal on Communications, 2022, 43(11): 44-52.
[15]	Jinyin CHEN, Shulong HU, Changyou XING, Guomin ZHANG. Deception defense method against intelligent penetration attack [J]. Journal on Communications, 2022, 43(10): 106-120.