工业物联网中基于Sarsa算法的节能计算卸载方案

doi:10.11959/j.issn.2096-3750.2022.00280

摘要/Abstract

摘要：

针对任务有完成截止时间要求的工业物联网系统，为了降低该类系统的总能源损耗，提出一种基于Sarsa算法的节能计算卸载方案。基于Sarsa算法的特点对能耗优化问题进行了解耦，由Sarsa算法进行外部状态值函数的迭代，选择合适的边缘计算服务器进行计算任务卸载，再由粒子群优化算法解决其中的资源分配问题，最后更新环境信息进行下一轮算法迭代，直至算法满足结束条件。仿真结果表明，本方案具有更快的收敛速度并且有效降低了系统能耗。

关键词: 工业物联网, 边缘计算, Sarsa, 节能计算卸载, 资源分配

Abstract:

In order to reduce the total energy consumption of industrial internet of things systems with deadline requirements, a computing offloading scheme based on Sarsa algorithm was proposed.Based on the characteristics of Sarsa algorithm, the energy consumption optimization problem was coupled.The Sarsa algorithm iterated the external state value function, selected the appropriate edge computing server to offload the computing task, and then the particle swarm optimization algorithm solved the resource allocation problem.Finally, the environmental information was updated for the next round of algorithm iteration until the algorithm met the end conditions.The simulation results show that this scheme has faster convergence speed and effectively reduces the system energy consumption.

Key words: industrial internet of things, edge computing, Sarsa, energy-saving computation offloading, resource allocation

中图分类号:

TN929.5

孙君, 赵尚维康. 工业物联网中基于Sarsa算法的节能计算卸载方案[J]. 物联网学报, 2022, 6(3): 82-90.

Jun SUN, Shangweikang ZHAO. Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things[J]. Chinese Journal on Internet of Things, 2022, 6(3): 82-90.

图/表 6

图1

图2

表1

仿真主要参数"

参数项	参数值
工业设备数N	N=20
小区内部署的MEC服务器数量M	M=5
工业设备的计算能力 $f_{j}^{l}$	$f_{j}^{l} \sim U (0.5, 1.5) \times 10^{10} cycle/s$
MEC服务器的计算能力 $f_{i, \max}^{c}$	$f_{i, \max}^{c} = 5 \times 10^{10} cycle/s$
任务数据量的大小d_j	$d_{j} \sim U (1, 10) Mbit$
计算任务被完成所需的计算周期数c_j	$c_{j} \sim U (500, 1500) cycle/bit$
计算任务完成的截止时间 $t_{j}^{dl}$	$t_{j}^{dl} \sim U (10, 50) ms$
带宽B	B=10 MHz
设备的传输功率 p_j	$p_{j} = 500 mW$
设备等待回传结果时的待机功率 p_s	$p_{s} = 100 mW$
路径损耗因子v	V =4
高斯白噪声功率σ²	$σ^{2} = 2 \times 10^{- 13} W$

表1

图3

图4

图5

参考文献 36

[1]	MAO W L , ZHAO Z W , CHANG Z ,et al. Energy-efficient industrial Internet of things,overview and open issues[J]. IEEE Transactions on Industrial Informatics, 2021,17(11): 7225-7237.
[2]	姚鑫 . 软件定义工业物联网计算卸载技术研究[D]. 成都,电子科技大学, 2019.
	YAO X . A research on software-defined industrial Internet of things computing offloading technology[D]. Chengdu,University of Electronic Science and Technology of China, 2019.
[3]	CHEN B T , WAN J F , SHU L ,et al. Smart factory of industry 4.0,key technologies,application case,and challenges[J]. IEEE Access, 2018,6: 6505-6519.
[4]	YOUNAN M N , HOUSSEIN E H , ELHOSENY M ,et al. Challenges and recommended technologies for the industrial Internet of things,a comprehensive review[J]. Measurement, 2020,151,107198.
[5]	刘斐, 曹钰杰, 章国安 . 车联网场景下移动边缘计算协作式资源分配策略[J]. 电讯技术, 2021,61(7): 858-864.
	LIU F , CAO Y J , ZHANG G A . Collaborative resource allocation strategy for mobile edge computing in vehicular networks[J]. Telecommunication Engineering, 2021,61(7): 858-864.
[6]	代振楠 . 工业互联网中基于边缘计算的任务卸载策略研究[D]. 长沙:湖南大学, 2019.
	DAI Z N . Research on task offloading strategy based on edge computing for industrial Internet[D]. Changsha,Hunan University, 2019.
[7]	DUAN X T , XU F,SUNYY . Research on offloading strategy in edge computing of Internet of things[C]// Proceedings of 2020 International Conference on Computer Network,Electronic and Automation (ICCNEA). Piscataway,IEEE Press, 2020: 206-210.
[8]	HOU X W , REN Z Y , YANG K ,et al. IIoT-MEC,a novel mobile edge computing framework for 5G-enabled IIoT[C]// Proceedings of 2019 IEEE Wireless Communications and Networking Conference. Piscataway,IEEE Press, 2019: 1-7.
[9]	MENG J Y , TAN H S , XU C ,et al. Dedas,online task dispatching and scheduling with bandwidth constraint in edge computing[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. Piscataway,IEEE Press, 2019: 2287-2295.
[10]	LI M , YU F R , SI P B ,et al. Energy-efficient machine-to-machine (M2M) communications in virtualized cellular networks with mobile edge computing (MEC)[J]. IEEE Transactions on Mobile Computing, 2019,18(7): 1541-1555.
[11]	JIANG C F , CHENG X L , GAO H H ,et al. Toward computation offloading in edge computing,asurvey[J]. IEEE Access, 7: 131543-131558.
[12]	HE X , DOU W C . Offloading deadline-aware task in edge computing[C]// Proceedings of 2020 IEEE 13th International Conference on Cloud Computing. Piscataway,IEEE Press, 2020: 28-30.
[13]	黄永明, 郑冲, 张征明 ,等. 大规模无线通信网络移动边缘计算和缓存研究[J]. 通信学报, 2021,42(4): 44-61.
	HUANG Y M , ZHENG C , ZHANG Z M ,et al. Research on mobile edge computing and caching in massive wireless communication network[J]. Journal on Communications, 2021,42(4): 44-61.
[14]	LI Z , ZHOU X , QIN Y F . A survey of mobile edge computing in the industrial Internet[C]// Proceedings of 2019 7th International Conference on Information,Communication and Networks (ICICN). Piscataway,IEEE Press, 2019: 94-98.
[15]	安宜豪 . 移动边缘计算中面向低功耗的任务迁移问题研究[D]. 哈尔滨,哈尔滨工业大学, 2019.
	AN Y H . Research on task offloading for low-power in MEC[D]. Harbin,Harbin Institute of Technology, 2019.
[16]	LYU X C , TIAN H , SENGUL C ,et al. Multiuser joint task offloading and resource optimization in proximate clouds[J]. IEEE Transactions on Vehicular Technology, 2017,66(4): 3435-3447.
[17]	WANG C M , LIANG CC , YU F R ,et al. Computation offloading and resource allocation in wireless cellular networks with mobile edge computing[J]. IEEE Transactions on Wireless Communications, 2017,16(8): 4924-4938.
[18]	BOYD S . Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends? in Machine Learning, 2010,3(1): 1-122.
[19]	YOU C S , HUANG K B , CHAE H ,et al. Energy-efficient resource allocation for mobile-edge computation offloading[J]. IEEE Transactions on Wireless Communications, 2017,16(3): 1397-1411.
[20]	EL HABER E , NGUYEN T M , ASSI C ,et al. An energy-efficient task offloading solution for MEC-based IoT in ultra-dense networks[C]// Proceedings of 2019 IEEE Wireless Communications and Networking Conference. Piscataway,IEEE Press, 2019: 1-7.
[21]	LI Q P , ZHAO J H , GONG Y ,et al. Energy-efficient computation offloading and resource allocation in fog computing for Internet of Everything[J]. China Communications, 2019,16(3): 32-41.
[22]	季子豪, 江凌云 . 一种基于遗传算法的多站点协同计算卸载算法[J]. 计算机工程与科学, 2021,43(3): 426-434.
	JI Z H , JIANG L Y . A genetic-based multi-site collaborative computation offloading algorithm[J]. Computer Engineering ＆Science, 2021,43(3): 426-434.
[23]	KE H C , WANG J , WANG H ,et al. Joint optimization of data offloading and resource allocation with renewable energy aware for IoT devices,a deep reinforcement learning approach[J]. IEEE Access, 7: 179349-179363.
[24]	张延华, 杨乐, 李萌 ,等. 基于 Q-learning 的工业互联网资源优化调度[J]. 北京工业大学学报, 2020,46(11): 1213-1221.
	ZHANG Y H , YANG L , LI M ,et al. Optimization of resource allocation for industrial Internet based on Q-learning[J]. Journal of Beijing University of Technology, 2020,46(11): 1213-1221.
[25]	石雪琴 . MEC 网络中联合优化计算卸载算法研究[D]. 重庆,重庆邮电大学, 2020.
	SHI X Q . Research on joint optimization of computation offloading in MEC network[D]. Chongqing,Chongqing University of Posts and Telecommunications, 2020.
[26]	程百川 . 基于深度强化学习的 MEC 计算卸载和资源分配研究[D]. 北京,北京邮电大学, 2019.
	CHENG B C . Research on MEC computing offloading and resource allocation based on deep reinforcement learning[D]. Beijing,Beijing University of Posts and Telecommunications, 2019.
[27]	李季 . 基于深度强化学习的移动边缘计算中的计算卸载与资源分配算法研究与实现[D]. 北京,北京邮电大学, 2019.
	LI J . Research and implementation of computation offloading and resource allocation algorithm in mobile edge computing based on deep reinforcement learning[D]. Beijing,Beijing University of Posts and Telecommunications, 2019.
[28]	李超 . 基于深度强化学习的移动边缘计算卸载策略及其物理层安全研究[D]. 广州,广州大学, 2020.
	LI C . Research on mobile edge unloading strategy based on deep reinforcement learning and its physical layer security[D]. Guangzhou:Guangzhou University, 2020.
[29]	KHALIL R A , SAEED N , MASOOD M ,et al. Deep learning in the industrial Internet of Things,potentials,challenges,and emerging applications[J]. IEEE Internet of Things Journal, 2021,8(14): 11016-11040.
[30]	黄冬晴 . 移动边缘计算中联合计算卸载和资源分配策略研究[D]. 上海,华东师范大学, 2021.
	HUANG D Q . Research on joint computation offloading and resource allocation strategy in mobile edge computing[D]. Shanghai,East China Normal University, 2021.
[31]	ALFAKIH T , HASSAN M M , GUMAEI A ,et al. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J]. IEEE Access, 8: 54074-54084.
[32]	LI Y Y,FADDAE , MANERBA D , et al . Reinforcement learning algorithms for online single-machine scheduling[C]// Proceedings of the 2020 Federated Conference on Computer Science and Information Systems","Annals of Computer Science and Information Systems. Piscataway,IEEE Press, 2020: 277-283.
[33]	AHSAN W , YI W Q , QIN Z J ,et al. Resource allocation in uplink NOMA-IoT networks,a reinforcement-learning approach[J]. IEEE Transactions on Wireless Communications, 2021,20(8): 5083-5098.
[34]	JIANG H , GUI R J , CHEN Z ,et al. An improved sarsa(-) reinforcement learning algorithm for wireless communication systems[J]. IEEE Access, 7: 115418-115427.
[35]	KUCHIBHOTLA V , HARSHITHA P , GOYAL S . An N-step look ahead algorithm using mixed (on and off) policy reinforcement learning[C]// Proceedings of 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS). Piscataway,IEEE Press, 2020: 677-681.
[36]	KENNEDY J , EBERHART R . Particle swarm optimization[C]// Proc eedings of ICNN'95 - International Conference on Neural Networks. Piscataway,IEEE Press, 1995: 1942-1948.