基于深度强化学习的能源互联网智能巡检任务分配机制

doi:10.11959/j.issn.1000-436x.2021071

Abstract

Abstract:

In order to reduce the cost and improve efficiency of power line inspection, UAV (unmanned aerial vehicle), which use mobile edge computing technology to access and process service data, are used to inspect power lines in the energy internet.However, due to the dynamic changes of UAV data transmission demand and geographical location, the edge server load will be unbalanced, which causes higher service processing delay and network energy consumption.Thus, an intelligent inspection task allocation mechanism for energy internet based on deep reinforcement learning was proposed.First, a two-layer edge network task offloading model was established to archive joint optimization of multi-objectives, such as delay and energy consumption.It was designed by comprehensively considering the route of UAV and edge nodes, different demands of services and limited service capabilities of edge nodes.Furthermore, based on Lyapunov optimization theory and dual-time-scaled mechanism, proximal policy optimization algorithm based deep reinforcement learning was used to solve the connection relationship and offloading strategy of edge servers between fixed edge sink layer and mobile edge access layer.The simulation results show that, the proposed mechanism can reduce the service request delay and system energy consumption while ensuring the stability of system.

Key words: patrol UAV, ask offloading, proximal policy optimization, Lyapunov optimization, artificial intelligence

CLC Number:

TN92

Siya XU, Yifei XING, Shaoyong GUO, Chao YANG, Xuesong QIU, Luoming MENG. Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet[J]. Journal on Communications, 2021, 42(5): 191-204.

Figures/Tables 10

参数	含义
$t = {t_{1}, t_{2}, \dots, t_{n}}$	短时隙集合
$T = {T_{1}, T_{2}, \dots, T_{m}}$	长时隙集合
$U = {u_{1}, \dots, u_{i}, \dots, u_{x}}$	UAV的集合
$S = {s_{1}, \dots, s_{j}, \dots, s_{y}}$	Sub-MECS的集合
$M = {m_{1}, \dots, m_{k}, \dots, m_{z}}$	M-MECS的集合
$R_{i}^{u} (t)$	u _i在时隙t产生的任务请求
$x^{h} (t)$	时隙t的计算卸载策略
$λ^{sub} (t) = {λ_{1}^{sub} (t), \dots, λ_{y}^{sub} (t)}$	Sub-MECS 在时隙t的任务到达率
$λ^{main} (t) = {λ_{1}^{main} (t), \dots, λ_{z}^{main} (t)}$	M-MECS 在时隙t 的任务到达率
$P_{i}^{u, c}$	每时隙中UAV的运行功耗
$E_{i}^{u s} (t) 和 E_{i}^{u m} (t)$	UAV在时隙t向Sub-MECS和M-MECS的传输能耗
$E_{j}^{s, c} (t)$	Sub-MECS在时隙t处理能耗
$P_{k}^{m, c} (t)$	M-MECS在时隙t的运行功耗
$D_{i}^{u s} (t) 和 D_{i}^{u m} (t)$	UAV在时隙t向Sub-MECS和M-MECS的传输时延
$D_{j}^{s, c} (t)$	Sub-MECS在时隙t处理时延
$D_{k}^{m, c} (t)$	M-MECS在时隙t处理时延
$D_{k}^{cloud} (t)$	M-MECS在时隙云传输时延

References 32

[1]	李君海, 张苗苗, 熊道洋 . 基于实时信息传输技术的无人机巡检管控平台[J]. 测绘与空间地理信息, 2020,43(6): 165-167,171.
	LI J H , ZHANG M M , XIONG D Y . An UAV power inspection management platform based on real-time message transmission technology[J]. Geomatics ＆ Spatial Information Technology, 2020,43(6): 165-167,171.
[2]	陈兰波 . 电力线路无人机巡检方案研究[J]. 科技与创新, 2020(11): 36-38,41.
	CHEN L B . Research on UAV patrol scheme of power line[J]. Science and Technology ＆ Innovation, 2020(11): 36-38,41.
[3]	严波, 林世忠, 张振威 ,等. 无人机电力巡检技术应用分析[J]. 自动化应用, 2019(12): 155-156.
	YAN B , LIN S Z , ZHANG Z W ,et al. Application analysis of UAV power inspection technology[J]. Automation Application, 2019(12): 155-156.
[4]	HAN D S , LI S J , CHEN Z X . Hybrid energy ratio allocation algorithm in a multi-base-station collaboration system[J]. IEEE Access, 2019,7: 147001-147009.
[5]	JEONG S , SIMEONE O , KANG J . Mobile edge computing via a UAV-mounted cloudlet:optimization of bit allocation and path planning[J]. IEEE Transactions on Vehicular Technology, 2018,67(3): 2049-2063.
[6]	WU Q Q , ZENG Y , ZHANG R . Joint trajectory and communication design for multi-UAV enabled wireless networks[J]. IEEE Transactions on Wireless Communications, 2018,17(3): 2109-2121.
[7]	WEI X L , TANG C G , FAN J H ,et al. Joint optimization of energy consumption and delay in cloud-to-thing continuum[J]. IEEE Internet of Things Journal, 2019,6(2): 2325-2337.
[8]	HU X Y , WONG K K , YANG K ,et al. UAV-assisted relaying and edge computing:scheduling and trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2019,18(10): 4738-4752.
[9]	HUA M , WANG Y , ZHANG Z M ,et al. Power-efficient communication in UAV-aided wireless sensor networks[J]. IEEE Communications Letters, 2018,22(6): 1264-1267.
[10]	ZHANG T K , XU Y , LOO J ,et al. Joint computation and communication design for UAV-assisted mobile edge computing in IoT[J]. IEEE Transactions on Industrial Informatics, 2020,16(8): 5505-5516.
[11]	ATEYA A A , MUTHANNA A , KIRICHEK R ,et al. Energy-and latency-aware hybrid offloading algorithm for UAVs[J]. IEEE Access, 2019,7: 37587-37600.
[12]	王立科 . 无人机技术在电力巡检信息化管理中的应用研究[J]. 机电信息, 2019(27): 88-89.
	WANG L K . Research on application of UAV technology in information management of power patrol inspection[J]. Mechanical and Electrical Information, 2019(27): 88-89.
[13]	马青岷 . 无人机电力巡检及三维模型重建技术研究[D]. 济南:山东大学, 2017.
	MA Q M . Research of powerline inspection based on UAV and 3D model technology[D]. Jinan:Shandong University, 2017.
[14]	SHUO W , XING Z , ZHI Y ,et al. Cooperative edge computing with sleep control under nonuniform traffic in mobile edge networks[J]. IEEE Internet of Things Journal, 2019,6(3): 4295-4306.
[15]	SHAW J A . Radiometry and the Friis transmission equation[J]. American Journal of Physics, 2013,81(1): 33-37.
[16]	DOWNEY C . Understanding wireless range calculations[J]. Electronic Design, 2013.
[17]	MAO Y Y , ZHANG J , SONG S H ,et al. Power-delay tradeoff in multi-user mobile-edge computing systems[C]// 2016 IEEE Global Communications Conference. Piscataway:IEEE Press, 2016: 1-6.
[18]	BURD T D , BRODERSEN R W . Processor design for portable systems[J]. Journal of VLSI Signal Processing Systems for Signal,Image and Video Technology, 1996,13(2/3): 203-221.
[19]	FAN X B , WEBER W D , BARROSO L A . Power provisioning for a warehouse-sized computer[J]. ACM SIGARCH Computer Architecture News, 2007,35(2): 13-23.
[20]	CHEN L X , ZHOU S , XU J . Computation peer offloading for energy-constrained mobile edge computing in small-cell networks[J]. IEEE/ACM Transactions on Networking, 2018,26(4): 1619-1632.
[21]	YANG L , CAO J N , TANG S J ,et al. A framework for partitioning and execution of data stream applications in mobile cloud computing[C]// 2012 IEEE Fifth International Conference on Cloud Computing. Piscataway:IEEE Press, 2012: 794-802.
[22]	CHEN L X , XU J , ZHOU S . Computation peer offloading in mobile edge computing with energy budgets[C]// GLOBECOM 2017 - 2017 IEEE Global Communications Conference. Piscataway:IEEE Press, 2017: 1-6.
[23]	NEELY J M . Stochastic network optimization with application to communication and queueing systems[J]. Synthesis Lectures on Communication Networks, 2010,3(1): 211.
[24]	XU S Y , LIU Q C , GONG B ,et al. RJCC:reinforcement-learningbased joint communicational-and-computational resource allocation mechanism for smart city IoT[J]. IEEE Internet of Things Journal, 2020,7(9): 8059-8076.
[25]	HEESS N , DHRUVA T B , SRIRAM S ,et al. Emergence of locomotion behaviours in rich environments[J]. arXiv Preprint,arXiv:1707.02286, 2017.
[26]	SCHULMAN J , WOLSKI F , DHARIWAL P ,et al. Proximal policy optimization algorithms[J]. arXiv Preprint,arXiv:1707.06347, 2017.
[27]	王志夫 . 基于深度强化学习的双足机器人步行运动控制[D]. 济南:山东大学, 2020.
	WANG Z F . Deep reinforcement learning based walking control of biped robot[D]. Jinan:Shandong University, 2020.
[28]	王鸿涛 . 基于强化学习的机械臂自学习控制[D]. 哈尔滨:哈尔滨工业大学, 2019.
	WANG H T . Self learning control of mechanical arm based on reinforcement learning[D]. Harbin:Harbin Institute of Technology, 2019.
[29]	胡彦娟 . 移动边缘计算中任务卸载及资源分配算法研究[D]. 重庆:重庆邮电大学, 2020.
	HU Y J . Research on task offloading and resource allocation algorithm in mobile edge computing[D]. Chongqing:Chongqing University of Posts and Telecommunications, 2020.
[30]	GAO Z B , WEN B , HUANG L F ,et al. Q-learning-based power control for LTE enterprise femtocell networks[J]. IEEE Systems Journal, 2017,11(4): 2699-2707.
[31]	LI R , ZHANG C Y , PATRAS P ,et al. Learning driven mobility control of airborne base stations in emergency networks[J]. ACM SIGMETRICS Performance Evaluation Review, 2019,46(3): 163-166.
[32]	KWAK J , KIM Y , LEE J ,et al. DREAM:dynamic resource and task allocation for energy minimization in mobile cloud systems[J]. IEEE Journal on Selected Areas in Communications, 2015,33(12): 2510-2523.

Metrics

Recommended 0

No Suggested Reading articles found!

Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 32

Related Articles 15

Metrics

Recommended 0

[1]	Feibo JIANG, Yubo PENG, Li DONG. Deep image semantic communication model for 6G [J]. Journal on Communications, 2023, 44(3): 198-208.
[2]	Jingyu WANG, Zirui ZHUANG. Research on a knowledge-defined polymorphic network attainable service architecture [J]. Journal on Communications, 2022, 43(4): 71-82.
[3]	Zhisheng NIU, Sheng ZHOU, Yuxuan SUN. Green communication and networking for Carbon-peaking and Carbon-neutrality: challenges and solutions [J]. Journal on Communications, 2022, 43(2): 1-14.
[4]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[5]	Pin LYU, Jia XU, Taoshen LI, Wenbiao XU. Survey on edge computing technology for autonomous driving [J]. Journal on Communications, 2021, 42(3): 190-208.
[6]	Chuanhong LIU, Caili GUO, Yang YANG, Chunyan FENG, Qizheng SUN, Jiujiu CHEN. Intelligent task-oriented semantic communication method in artificial intelligence of things [J]. Journal on Communications, 2021, 42(11): 97-108.
[7]	Tao HUANG, Jiang LIU, Shuo WANG, Chen ZHANG, Yunjie LIU. Survey of the future network technology and trend [J]. Journal on Communications, 2021, 42(1): 130-150.
[8]	Yusun FU,Genke YANG. Application of artificial intelligence in mobile communication:challenge and practice [J]. Journal on Communications, 2020, 41(9): 190-201.
[9]	Yingchang LIANG,Junjie TAN,Dusit Niyato. Overview on intelligent wireless communication technology [J]. Journal on Communications, 2020, 41(7): 1-17.
[10]	Lindong ZHAO,Wenqin ZHUANG,Jianxin CHEN,Liang ZHOU. Hierarchical task offloading in heterogeneous cellular network:modeling and optimization [J]. Journal on Communications, 2020, 41(4): 34-44.
[11]	Yan LIN, Shuai YAN, Yijin ZHANG, Chunguo LI, Feng SHU. Flow-of-traffic prediction program based mobile edge computing for Internet of vehicles using double auction [J]. Journal on Communications, 2020, 41(12): 205-214.
[12]	Gangxiang SHEN. Application research of optical communication network based on artificial intelligence technique [J]. Journal on Communications, 2020, 41(1): 162-168.
[13]	Hongyu SUN,Yuan HE,Jice WANG,Ying DONG,Lipeng ZHU,He WANG,Yuqing ZHANG. Application of artificial intelligence technology in the field of security vulnerability [J]. Journal on Communications, 2018, 39(8): 1-17.
[14]	Ping ZHANG,Huiyong LIU,Wenjing LI,Fanqin ZHOU. Industrial intelligent network: deepening and upgrading of industrial Internet [J]. Journal on Communications, 2018, 39(12): 134-140.
[15]	Liang WEI,Tao HUANG,Jiao ZHANG,Zenan WANG,Jiang LIU,Yunjie LIU. Service chain mapping algorithm based on reinforcement learning [J]. Journal on Communications, 2018, 39(1): 90-100.