工业物联网中基于Sarsa算法的节能计算卸载方案

doi:10.11959/j.issn.2096-3750.2022.00280

Abstract

Abstract:

In order to reduce the total energy consumption of industrial internet of things systems with deadline requirements, a computing offloading scheme based on Sarsa algorithm was proposed.Based on the characteristics of Sarsa algorithm, the energy consumption optimization problem was coupled.The Sarsa algorithm iterated the external state value function, selected the appropriate edge computing server to offload the computing task, and then the particle swarm optimization algorithm solved the resource allocation problem.Finally, the environmental information was updated for the next round of algorithm iteration until the algorithm met the end conditions.The simulation results show that this scheme has faster convergence speed and effectively reduces the system energy consumption.

Key words: industrial internet of things, edge computing, Sarsa, energy-saving computation offloading, resource allocation

CLC Number:

TN929.5

Jun SUN, Shangweikang ZHAO. Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things[J]. Chinese Journal on Internet of Things, 2022, 6(3): 82-90.

Figures/Tables 6

参数项	参数值
工业设备数N	N=20
小区内部署的MEC服务器数量M	M=5
工业设备的计算能力 $f_{j}^{l}$	$f_{j}^{l} \sim U (0.5, 1.5) \times 10^{10} cycle/s$
MEC服务器的计算能力 $f_{i, \max}^{c}$	$f_{i, \max}^{c} = 5 \times 10^{10} cycle/s$
任务数据量的大小d_j	$d_{j} \sim U (1, 10) Mbit$
计算任务被完成所需的计算周期数c_j	$c_{j} \sim U (500, 1500) cycle/bit$
计算任务完成的截止时间 $t_{j}^{dl}$	$t_{j}^{dl} \sim U (10, 50) ms$
带宽B	B=10 MHz
设备的传输功率 p_j	$p_{j} = 500 mW$
设备等待回传结果时的待机功率 p_s	$p_{s} = 100 mW$
路径损耗因子v	V =4
高斯白噪声功率σ²	$σ^{2} = 2 \times 10^{- 13} W$

References 36

[1]	MAO W L , ZHAO Z W , CHANG Z ,et al. Energy-efficient industrial Internet of things,overview and open issues[J]. IEEE Transactions on Industrial Informatics, 2021,17(11): 7225-7237.
[2]	姚鑫 . 软件定义工业物联网计算卸载技术研究[D]. 成都,电子科技大学, 2019.
	YAO X . A research on software-defined industrial Internet of things computing offloading technology[D]. Chengdu,University of Electronic Science and Technology of China, 2019.
[3]	CHEN B T , WAN J F , SHU L ,et al. Smart factory of industry 4.0,key technologies,application case,and challenges[J]. IEEE Access, 2018,6: 6505-6519.
[4]	YOUNAN M N , HOUSSEIN E H , ELHOSENY M ,et al. Challenges and recommended technologies for the industrial Internet of things,a comprehensive review[J]. Measurement, 2020,151,107198.
[5]	刘斐, 曹钰杰, 章国安 . 车联网场景下移动边缘计算协作式资源分配策略[J]. 电讯技术, 2021,61(7): 858-864.
	LIU F , CAO Y J , ZHANG G A . Collaborative resource allocation strategy for mobile edge computing in vehicular networks[J]. Telecommunication Engineering, 2021,61(7): 858-864.
[6]	代振楠 . 工业互联网中基于边缘计算的任务卸载策略研究[D]. 长沙:湖南大学, 2019.
	DAI Z N . Research on task offloading strategy based on edge computing for industrial Internet[D]. Changsha,Hunan University, 2019.
[7]	DUAN X T , XU F,SUNYY . Research on offloading strategy in edge computing of Internet of things[C]// Proceedings of 2020 International Conference on Computer Network,Electronic and Automation (ICCNEA). Piscataway,IEEE Press, 2020: 206-210.
[8]	HOU X W , REN Z Y , YANG K ,et al. IIoT-MEC,a novel mobile edge computing framework for 5G-enabled IIoT[C]// Proceedings of 2019 IEEE Wireless Communications and Networking Conference. Piscataway,IEEE Press, 2019: 1-7.
[9]	MENG J Y , TAN H S , XU C ,et al. Dedas,online task dispatching and scheduling with bandwidth constraint in edge computing[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. Piscataway,IEEE Press, 2019: 2287-2295.
[10]	LI M , YU F R , SI P B ,et al. Energy-efficient machine-to-machine (M2M) communications in virtualized cellular networks with mobile edge computing (MEC)[J]. IEEE Transactions on Mobile Computing, 2019,18(7): 1541-1555.
[11]	JIANG C F , CHENG X L , GAO H H ,et al. Toward computation offloading in edge computing,asurvey[J]. IEEE Access, 7: 131543-131558.
[12]	HE X , DOU W C . Offloading deadline-aware task in edge computing[C]// Proceedings of 2020 IEEE 13th International Conference on Cloud Computing. Piscataway,IEEE Press, 2020: 28-30.
[13]	黄永明, 郑冲, 张征明 ,等. 大规模无线通信网络移动边缘计算和缓存研究[J]. 通信学报, 2021,42(4): 44-61.
	HUANG Y M , ZHENG C , ZHANG Z M ,et al. Research on mobile edge computing and caching in massive wireless communication network[J]. Journal on Communications, 2021,42(4): 44-61.
[14]	LI Z , ZHOU X , QIN Y F . A survey of mobile edge computing in the industrial Internet[C]// Proceedings of 2019 7th International Conference on Information,Communication and Networks (ICICN). Piscataway,IEEE Press, 2019: 94-98.
[15]	安宜豪 . 移动边缘计算中面向低功耗的任务迁移问题研究[D]. 哈尔滨,哈尔滨工业大学, 2019.
	AN Y H . Research on task offloading for low-power in MEC[D]. Harbin,Harbin Institute of Technology, 2019.
[16]	LYU X C , TIAN H , SENGUL C ,et al. Multiuser joint task offloading and resource optimization in proximate clouds[J]. IEEE Transactions on Vehicular Technology, 2017,66(4): 3435-3447.
[17]	WANG C M , LIANG CC , YU F R ,et al. Computation offloading and resource allocation in wireless cellular networks with mobile edge computing[J]. IEEE Transactions on Wireless Communications, 2017,16(8): 4924-4938.
[18]	BOYD S . Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends? in Machine Learning, 2010,3(1): 1-122.
[19]	YOU C S , HUANG K B , CHAE H ,et al. Energy-efficient resource allocation for mobile-edge computation offloading[J]. IEEE Transactions on Wireless Communications, 2017,16(3): 1397-1411.
[20]	EL HABER E , NGUYEN T M , ASSI C ,et al. An energy-efficient task offloading solution for MEC-based IoT in ultra-dense networks[C]// Proceedings of 2019 IEEE Wireless Communications and Networking Conference. Piscataway,IEEE Press, 2019: 1-7.
[21]	LI Q P , ZHAO J H , GONG Y ,et al. Energy-efficient computation offloading and resource allocation in fog computing for Internet of Everything[J]. China Communications, 2019,16(3): 32-41.
[22]	季子豪, 江凌云 . 一种基于遗传算法的多站点协同计算卸载算法[J]. 计算机工程与科学, 2021,43(3): 426-434.
	JI Z H , JIANG L Y . A genetic-based multi-site collaborative computation offloading algorithm[J]. Computer Engineering ＆Science, 2021,43(3): 426-434.
[23]	KE H C , WANG J , WANG H ,et al. Joint optimization of data offloading and resource allocation with renewable energy aware for IoT devices,a deep reinforcement learning approach[J]. IEEE Access, 7: 179349-179363.
[24]	张延华, 杨乐, 李萌 ,等. 基于 Q-learning 的工业互联网资源优化调度[J]. 北京工业大学学报, 2020,46(11): 1213-1221.
	ZHANG Y H , YANG L , LI M ,et al. Optimization of resource allocation for industrial Internet based on Q-learning[J]. Journal of Beijing University of Technology, 2020,46(11): 1213-1221.
[25]	石雪琴 . MEC 网络中联合优化计算卸载算法研究[D]. 重庆,重庆邮电大学, 2020.
	SHI X Q . Research on joint optimization of computation offloading in MEC network[D]. Chongqing,Chongqing University of Posts and Telecommunications, 2020.
[26]	程百川 . 基于深度强化学习的 MEC 计算卸载和资源分配研究[D]. 北京,北京邮电大学, 2019.
	CHENG B C . Research on MEC computing offloading and resource allocation based on deep reinforcement learning[D]. Beijing,Beijing University of Posts and Telecommunications, 2019.
[27]	李季 . 基于深度强化学习的移动边缘计算中的计算卸载与资源分配算法研究与实现[D]. 北京,北京邮电大学, 2019.
	LI J . Research and implementation of computation offloading and resource allocation algorithm in mobile edge computing based on deep reinforcement learning[D]. Beijing,Beijing University of Posts and Telecommunications, 2019.
[28]	李超 . 基于深度强化学习的移动边缘计算卸载策略及其物理层安全研究[D]. 广州,广州大学, 2020.
	LI C . Research on mobile edge unloading strategy based on deep reinforcement learning and its physical layer security[D]. Guangzhou:Guangzhou University, 2020.
[29]	KHALIL R A , SAEED N , MASOOD M ,et al. Deep learning in the industrial Internet of Things,potentials,challenges,and emerging applications[J]. IEEE Internet of Things Journal, 2021,8(14): 11016-11040.
[30]	黄冬晴 . 移动边缘计算中联合计算卸载和资源分配策略研究[D]. 上海,华东师范大学, 2021.
	HUANG D Q . Research on joint computation offloading and resource allocation strategy in mobile edge computing[D]. Shanghai,East China Normal University, 2021.
[31]	ALFAKIH T , HASSAN M M , GUMAEI A ,et al. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J]. IEEE Access, 8: 54074-54084.
[32]	LI Y Y,FADDAE , MANERBA D , et al . Reinforcement learning algorithms for online single-machine scheduling[C]// Proceedings of the 2020 Federated Conference on Computer Science and Information Systems","Annals of Computer Science and Information Systems. Piscataway,IEEE Press, 2020: 277-283.
[33]	AHSAN W , YI W Q , QIN Z J ,et al. Resource allocation in uplink NOMA-IoT networks,a reinforcement-learning approach[J]. IEEE Transactions on Wireless Communications, 2021,20(8): 5083-5098.
[34]	JIANG H , GUI R J , CHEN Z ,et al. An improved sarsa(-) reinforcement learning algorithm for wireless communication systems[J]. IEEE Access, 7: 115418-115427.
[35]	KUCHIBHOTLA V , HARSHITHA P , GOYAL S . An N-step look ahead algorithm using mixed (on and off) policy reinforcement learning[C]// Proceedings of 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS). Piscataway,IEEE Press, 2020: 677-681.
[36]	KENNEDY J , EBERHART R . Particle swarm optimization[C]// Proc eedings of ICNN'95 - International Conference on Neural Networks. Piscataway,IEEE Press, 1995: 1942-1948.

Metrics

Recommended 0

No Suggested Reading articles found!

Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 36

Related Articles 15

Metrics

Recommended 0

[1]	Zhihong WANG, Supeng LENG, Kai XIONG. Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing [J]. Chinese Journal on Internet of Things, 2023, 7(1): 18-26.
[2]	Bei TANG, Qian WANG, Siguang CHEN. Radio frequency energy harvesting-combined collaborative energy-saving computation offloading mechanism [J]. Chinese Journal on Internet of Things, 2023, 7(1): 83-92.
[3]	Peiyan YUAN, Saike SHAO, Ran WEI, Junna ZHANG, Xiaoyan ZHAO. Research on the cooperative offloading strategy of sensory data based on delay and energy constraints [J]. Chinese Journal on Internet of Things, 2023, 7(1): 109-117.
[4]	Yao LIU, Yueyuan HE, Hongjing ZHOU, Chaoliang LI, Chuang LI. Partial computation offloading method based on joint resource allocation for mobile edge computing [J]. Chinese Journal on Internet of Things, 2023, 7(1): 140-148.
[5]	Xian LI, Suzhi BI, Hongru ZENG, Bin LIN, Xiaohui LIN. Collaborative task offloading and resource allocation optimization for intelligent edge devices [J]. Chinese Journal on Internet of Things, 2022, 6(4): 41-52.
[6]	Lin SU, Xiaochao DANG, Zhanjun HAO, Chunrui RU, Xu SHANG. Dynamic adaptive offloading method based on WPT-MEC [J]. Chinese Journal on Internet of Things, 2022, 6(4): 128-138.
[7]	Nuo HUANG, Weijie LIU, Chen GONG. Industrial IoT oriented petahertz communication [J]. Chinese Journal on Internet of Things, 2022, 6(3): 37-46.
[8]	Meinan ZHANG, Mingqi ZHANG, Fei DING, Hengheng ZHUANG, Hairong MA. Offloading strategy with edge optimization of time delay and energy consumption in integrated satellite-terrestrial relay network [J]. Chinese Journal on Internet of Things, 2022, 6(3): 124-132.
[9]	Juan FANG, Zhiyuan YE, Mengyuan ZHANG, Jiamei SHI, Ziyi TENG. Research on elite hierarchical task offloading strategy based on reinforcement learning in edge-cloud collaboration scenario [J]. Chinese Journal on Internet of Things, 2022, 6(1): 91-100.
[10]	Xin SU, Su JIANG, Yiqing ZHOU. Research on location privacy protection of mobile terminals for maritime monitoring sensor networks [J]. Chinese Journal on Internet of Things, 2021, 5(4): 26-36.
[11]	Yongjun XU, Haoke YANG, Yinghui YE, Qianbin CHEN, Guangyue LU. A survey on resource allocation in backscatter communication networks [J]. Chinese Journal on Internet of Things, 2021, 5(3): 56-69.
[12]	Siqi SUN. Analysis and prospects of the development of the industrial Internet in the petrochemical industry [J]. Chinese Journal on Internet of Things, 2021, 5(3): 126-132.
[13]	Deze ZENG, Lvhao CHEN, Lin GU, Yuepeng LI. Cloud native based edge computing: vision and challenges [J]. Chinese Journal on Internet of Things, 2021, 5(2): 7-17.
[14]	Qi ZHANG, Yuna JIANG, Xiaohu GE, Yonghui LI. Resource allocation based on optimal transport theory in IoT edge computing [J]. Chinese Journal on Internet of Things, 2021, 5(2): 60-70.
[15]	Hao YUAN, Deke GUO, Guoming TANG, Lailong LUO. Online energy-aware task dispatching with QoS guarantee in edge computing [J]. Chinese Journal on Internet of Things, 2021, 5(2): 71-77.