SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究

doi:10.11959/j.issn.1000-0801.2024025

电信科学 ›› 2024, Vol. 40 ›› Issue (2): 96-106.doi: 10.11959/j.issn.1000-0801.2024025

• 研究与开发 • 上一篇

SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究

蒋守花, 王以伍

成都医学院现代教育技术中心，四川成都 610500

修回日期:2024-02-05 出版日期:2024-02-01 发布日期:2024-02-01
作者简介:蒋守花（1988- ），女，成都医学院现代教育技术中心工程师，主要研究方向为边缘计算、人工智能、物联网
王以伍（1982- ），男，成都医学院现代教育技术中心工程师，主要研究方向为数据挖掘、网络安全
基金资助:
四川省高等学校人文社会科学重点研究基地·四川省教育信息化应用与发展研究中心项目(JYXX23-002);成都医学院校基金科研项目(CYSYB23-02)

Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN

Shouhua JIANG, Yiwu WANG

Modern Education Technology Center, Chengdu Medical College, Chengdu 610500, China

Revised:2024-02-05 Online:2024-02-01 Published:2024-02-01
Supported by:
The Project of Sichuan Provincial Key Research Base of Humanities and Social Sciences and Sichuan Provincial Research Center for Application and Development of Education Informatization(JYXX23-002);The Chengdu Medical College Funded Research Project(CYSYB23-02)

摘要/Abstract

摘要：

随着网络技术的不断发展，基于Fat-Tree的网络拓扑结构分布式网络控制模式逐渐显露出其局限性，软件定义数据中心网络（software-defined data center network，SDCN）技术作为Fat-Tree网络拓扑的改进技术，受到越来越多研究者的关注。首先搭建了一个 SDCN 中的边缘计算架构和基于移动边缘计算（mobile edge computing，MEC）平台三层服务架构的任务卸载模型，结合移动边缘计算平台的实际应用场景，利用同策略经验回放和熵正则改进传统的深度Q网络（deep Q-leaning network，DQN）算法，优化了MEC平台的任务卸载策略，并设计了实验对基于同策略经验回放和熵正则的改进深度Q网络算法（improved DQN algorithm based on same strategy empirical playback and entropy regularization，RSS2E-DQN）和其他3种算法在负载均衡、能耗、时延、网络使用量几个方面进行对比分析，验证了改进算法在上述4个方面具有更优越的性能。

关键词: 软件定义数据中心网络, 深度强化学习, 边缘计算任务卸载, 同策略经验回放, 熵正则

Abstract:

With the continuous development of network technology, the network topology distributed network control mode based on Fat-Tree gradually reveals its limitations.Software-defined data center network (SDCN) technology, as an improved technology of Fat-Tree network topology, has attracted more and more researchers’ attention.Firstly, an edge computing architecture in SDCN and a task offloading model based on the three-layer service architecture of the mobile edge computing (MEC) platform were built, combined with the actual application scenarios of the MEC platform.Through the same strategy experience playback and entropy regularization, the traditional deep Q-leaning network (DQN) algorithm was improved, and the task offloading strategy of MEC platform was optimized.An improved DQN algorithm based on same strategy empirical playback and entropy regularization (RSS2E-DQN) was compared with three other algorithms in load balancing, energy consumption, delay and network usage.It is verified that the improved algorithm has better performance in the above four aspects.

Key words: software-defined data center network, deep reinforcement learning, edge computing task offloading, replay the same strategy experience, entropy regularity

中图分类号:

TP393

蒋守花, 王以伍. SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究[J]. 电信科学, 2024, 40(2): 96-106.

Shouhua JIANG, Yiwu WANG. Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN[J]. Telecommunications Science, 2024, 40(2): 96-106.

图/表 9

图1

图2

图3

图4

表1

图5

图6

图7

图8

参考文献 24

[1]	李丹, 陈贵海, 任丰原 ,等. 数据中心网络的研究进展与趋势[J]. 计算机学报, 2014,37(2): 259-274.
	LI D , CHEN G H , REN F Y ,et al. Data center network research progress and trends[J]. Chinese Journal of Computers, 2014,37(2): 259-274.
[2]	王斌锋, 苏金树, 陈琳 . 云计算数据中心网络设计综述[J]. 计算机研究与发展, 2016,53(9): 2085-2106.
	WANG B F , SU J S , CHEN L . Review of the design of data center network for cloud computing[J]. Journal of Computer Research and Development, 2016,53(9): 2085-2106.
[3]	NUNES B A A , MENDONCA M , NGUYEN X N ,et al. A survey of software-defined networking:past,present,and future of programmable networks[J]. IEEE Communications Surveys＆ Tutorials, 2014,16(3): 1617-1634.
[4]	吴强, 徐鑫, 刘国燕 . 基于SDN技术的数据中心基础网络构建[J]. 电信科学, 2013,29(1): 130-133,142.
	WU Q , XU X , LIU G Y . Construction of basic network in data center based on SDN technology[J]. Telecommunications Science, 2013,29(1): 130-133,142.
[5]	曹腾飞, 刘延亮, 王晓英 . 基于改进深度强化学习的边缘计算服务卸载算法[J]. 计算机应用, 2023,43(5): 1543-1550.
	CAO T F , LIU Y L , WANG X Y . Edge computing and service offloading algorithm based on improved deep reinforcement learning[J]. Journal of Computer Applications, 2023,43(5): 1543-1550.
[6]	卢海峰, 顾春华, 罗飞 ,等. 基于深度强化学习的移动边缘计算任务卸载研究[J]. 计算机研究与发展, 2020,57(7): 1539-1554.
	LU H F , GU C H , LUO F ,et al. Research on task offloading based on deep reinforcement learning in mobile edge computing[J]. Journal of Computer Research and Development, 2020,57(7): 1539-1554.
[7]	邝祝芳, 陈清林, 李林峰 ,等. 基于深度强化学习的多用户边缘计算任务卸载调度与资源分配算法[J]. 计算机学报, 2022,45(4): 812-824.
	KUANG Z F , CHEN Q L , LI L F ,et al. Multi-user edge computing task offloading scheduling and resource allocation based on deep reinforcement learning[J]. Chinese Journal of Computers, 2022,45(4): 812-824.
[8]	盛煜, 朱正伟, 朱晨阳 ,等. 基于深度强化学习的多目标边缘任务调度研究[J]. 电子测量技术, 2023,46(8): 74-81.
	SHENG Y , ZHU Z W , ZHU C Y ,et al. Research on multi-objective edge task scheduling based on deep reinforcement learning[J]. Electronic Measurement Technology, 2023,46(8): 74-81.
[9]	JAISWAL A , BABU A R , ZADEH M Z ,et al. A survey on contrastive self-supervised learning[J]. Technologies, 2020,9(1): 2.
[10]	GRAVES A , MOHAMED A R , HINTON G . Speech recognition with deep recurrent neural networks[C]// Proceedings of the 2013 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2013: 6645-6649.
[11]	SADIKI A , BENTAHAR J , DSSOULI R ,et al. Deep reinforcement learning for the computation offloading in MIMO-based Edge Computing[J]. Ad Hoc Networks, 2023(141): 103080.
[12]	LI M S , GAO J , ZHAO L ,et al. Deep reinforcement learning for collaborative edge computing in vehicular networks[J]. IEEE Transactions on Cognitive Communications and Networking, 2020,6(4): 1122-1135.
[13]	LI D J , XU S Y , LI P Y . Deep reinforcement learning-empowered resource allocation for mobile edge computing in cellular V2X networks[J]. Sensors, 2021,21(2): 372.
[14]	XIAO Z , DAI X X , JIANG H B ,et al. Vehicular task offloading via heat-aware MEC cooperation using game-theoretic method[J]. IEEE Internet of Things Journal, 2020,7(3): 2038-2052.
[15]	SHENG M , DAI Y P , LIU J Y ,et al. Delay-aware computation offloading in NOMA MEC under differentiated uploading delay[J]. IEEE Transactions on Wireless Communications, 2020,19(4): 2813-2826.
[16]	ZHAN W H , LUO C B , MIN G Y ,et al. Mobility-aware multi-user offloading optimization for mobile edge computing[J]. IEEE Transactions on Vehicular Technology, 2020,69(3): 3341-3356.
[17]	ALFAKIH T , HASSAN M M , GUMAEI A ,et al. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J]. IEEE Access, 2020 8: 54074-54084.
[18]	JO?ILO S , DáN G . Joint management of wireless and computing resources for computation offloading in mobile edge clouds[J]. IEEE Transactions on Cloud Computing, 2021,9(4): 1507-1520.
[19]	YUAN H T , ZHOU M C . Profit-maximized collaborative computation offloading and resource allocation in distributed cloud and edge computing systems[J]. IEEE Transactions on Automation Science and Engineering, 2021,18(3): 1277-1287.
[20]	ZHAO T C , ZHOU S , SONG L Q ,et al. Energy-optimal and delay-bounded computation offloading in mobile edge computing with heterogeneous clouds[J]. China Communications, 2020,17(5): 191-210.
[21]	KUANG Z F , LI L F , GAO J ,et al. Partial offloading scheduling and power allocation for mobile edge computing systems[J]. IEEE Internet of Things Journal, 2019,6(4): 6774-6785.
[22]	杨思明, 单征, 丁煜 ,等. 深度强化学习研究综述[J]. 计算机工程, 2021,47(12): 19-29.
	YANG S M , SHAN Z , DING Y ,et al. Survey of research on deep reinforcement learning[J]. Computer Engineering, 2021,47(12): 19-29.
[23]	ABADI M , AGARWAL A , BARHAM P ,et al. TensorFlow:large-scale machine learning on heterogeneous distributed systems[J]. arXiv prepnht, 2016,arXiv:1603.04467.
[24]	何荣希, 雷田颖, 林子薇 . 软件定义数据中心网络多约束节能路由算法[J]. 计算机研究与发展, 2019,56(6): 1219-1230.
	HE R X , LEI T Y , LIN Z W . Multi-constrained energy-saving routing algorithm in software-defined data center networks[J]. Journal of Computer Research and Development, 2019,56(6): 1219-1230.

符号	值	定义
C	1	云数据中心
E	70	边缘服务器
M	若干	移动智能设备
N	50	边缘设备发出任务
λ	0.9	折扣系数
W	10 Gbit/s	带宽
L	200	算法最大循环轮数
n	512	神经网络深度
A	150	随机抽取经验样本
p	(0,100)	卸载功率范围
X	300	动作空间
N₀	-180 dBm/Hz	环境高斯白噪声功率密度
F	10 Hz	边缘服务器总频率
P	4	算法数量
α	0.005	学习率

SDCN中基于深度强化学习的移动边缘计算任务卸载算法研究

Research on task offloading algorithm of mobile edge computing based on deep reinforcement learning in SDCN

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 24

相关文章 8

Metrics

推荐阅读 0

[1]	张维庭, 孙呈蕙, 王洪超, 代嘉宁. 算网资源智能适配与融合调度方法[J]. 电信科学, 2023, 39(9): 12-20.
[2]	刘峻朋, 夏玮玮, 刘晗, 修成林, 燕锋, 沈连丰. 面向电力业务质量保障的NR-U与Wi-Fi频谱共享[J]. 电信科学, 2023, 39(7): 11-22.
[3]	胡珈玮, 刘晓谦, 唐昕柯, 董宇涵. 基于DQN的UUV辅助水下无线光通信轨迹规划系统[J]. 电信科学, 2023, 39(5): 42-47.
[4]	廖熙雯, 冷甦鹏, 明昱君, 李天扬. 基于数字孪生的城市交通流智能预测与导引策略[J]. 电信科学, 2023, 39(3): 70-79.
[5]	康宇, 刘雅琼, 赵彤雨, 寿国础. AI算法在车联网通信与计算中的应用综述[J]. 电信科学, 2023, 39(1): 1-19.
[6]	邓丹昊, 王朝炜, 江帆, 王卫东. 无人机辅助无蜂窝大规模MIMO中的空地协同调度[J]. 电信科学, 2022, 38(8): 37-44.
[7]	伍仲丽, 曹园园, 黄文睿, 戴彬, 莫益军. 面向确定性网络的按需智能路由技术[J]. 电信科学, 2021, 37(11): 11-16.
[8]	桂飞,程阳,李丹,洪思虹. 互联网智能路由架构及算法[J]. 电信科学, 2020, 36(10): 12-20.