基于深度强化学习的海洋移动边缘计算卸载方法

doi:10.11959/j.issn.1000-436x.2022197

Abstract

Abstract:

The strong heterogeneity among the network nodes of the maritime information system brings complex and high-dimensional constraints for optimizing task offloading of the maritime mobile edge computing.The complex and diverse maritime applications also lead to the overload processing of computing tasks in local areas of the maritime network.In order to optimize the task offloading and resource management of maritime network, as well as meet the maritime application service requirements of low-latency and high-reliability, a hierarchical classification method of maritime nodes based on multi-layers attributes and a novel offloading method for maritime mobile edge computing based on deep reinforcement learning were proposed.Compared with conventional methods, simulation results show that the proposed method can effectively reduce the computing task offloading delay of the marine information system, and maintain the robustness of the maritime network with large-scale task flows.

Key words: maritime information system, edge computing, computing task offloading, power and computing resource allocation, deep reinforcement learning

CLC Number:

TN929.52

Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(10): 133-145.

Figures/Tables 13

参数	数值
系统总带宽/MHz	20
$L_{m}^{I}$ 节点处理能力/GHz	200
$L_{n}^{II}$ 节点处理能力/GHz	[10,50]
$L_{k}^{III}$ 节点处理能力/GHz	[2,3]
$L_{k}^{III}$ 节点数量/个	10
$L_{k}^{III}$ 节点输入数据的大小/MB	[10,30]
$L_{k}^{III}$ 节点完成计算内容的CPU周期数/GHz	[30,50]
上行链路最大发射功率/W	[0.05,0.1]
背景噪声功率/dBm	–70

References 33

[1]	瞿逢重, 来杭亮, 刘建章 ,等. 海洋物联网关键技术研究与应用[J]. 电信科学, 2021,37(7): 25-33.
	QU F Z , LAI H L , LIU J Z ,et al. Research and application on key techniques of marine IoT[J]. Telecommunications Science, 2021,37(7): 25-33.
[2]	YANG T , FENG H L , YANG C M ,et al. Multivessel computation offloading in maritime mobile edge computing network[J]. IEEE Internet of Things Journal, 2019,6(3): 4063-4073.
[3]	CHEN S Y , LUO S Y , WANGQ ,et al. An intelligent task offloading algorithm (iTOA) for UAV edge computing network[J]. Digital Communications and Networks, 2021,6(4): 433-443.
[4]	LIANG M Z , SU X , LIU X F ,et al. Intelligent ocean convergence platform based on IoT empowered with edge computing[J]. Journal of Internet Technology, 2020,21(1): 235-244.
[5]	ZHOU Y Q , LIU H , PAN Z G ,et al. Two-stage cooperative multicast transmission with optimized power consumption and guaranteed coverage[J]. IEEE Journal on Selected Areas in Communications, 2014,32(2): 274-284.
[6]	SHU W N , LI Y . Joint offloading strategy based on quantum particle swarm optimization for MEC-enabled vehicular networks[J]. Digital Communications and Networks, 2022:doi.org/10.1016/j.dcan.2022.03.009.
[7]	LIU L , ZHOU Y Q , ZHUANG W H ,et al. Tractable coverage analysis for hexagonal macrocell-based heterogeneous UDNs with adaptive interference-aware CoMP[J]. IEEE Transactions on Wireless Communications, 2019,18(1): 503-517.
[8]	ANBALAGAN S , KUMAR D , RAJA G ,et al. SDN assisted Stackelberg game model for LTE-WiFi offloading in 5G networks[J]. Digital Communications and Networks, 2019,5(4): 268-275.
[9]	EL-HABER E , NGUYEN T M , ASSI C . Joint optimization of computational cost and devices energy for task offloading in multi-tier edge-clouds[J]. IEEE Transactions on Communications, 2019,67(5): 3407-3421.
[10]	NING Z L , DONG P R , KONG X J ,et al. A cooperative partial computation offloading scheme for mobile edge computing enabled Internet of things[J]. IEEE Internet of Things Journal, 2019,6(3): 4804-4814.
[11]	JIANG F , WEI F M , WANG J X ,et al. Delay-aware energy minimization offloading scheme for mobile edge computing[C]// Proceedings of 2020 IEEE/CIC International Conference on Communications in China (ICCC). Piscataway:IEEE Press, 2020: 717-722.
[12]	HU X Y , WANG L F , WONG K K ,et al. Edge and central cloud computing:a perfect pairing for high energy efficiency and low-latency[J]. IEEE Transactions on Wireless Communications, 2020,19(2): 1070-1083.
[13]	ZHANG K , MAO Y M , LENG S P ,et al. Mobile-edge computing for vehicular networks:a promising network paradigm with predictive off-loading[J]. IEEE Vehicular Technology Magazine, 2017,12(2): 36-44.
[14]	DAI Y Y , XU D , MAHARJAN S ,et al. Joint load balancing and offloading in vehicular edge computing and networks[J]. IEEE Internet of Things Journal, 2019,6(3): 4377-4387.
[15]	HUY H V , HO T M , LE L B . Mobility-aware computation offloading in MEC-based vehicular wireless networks[J]. IEEE Communications Letters, 2020,24(2): 466-469.
[16]	DU J B , YU F R , CHU X L ,et al. Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization[J]. IEEE Transactions on Vehicular Technology, 2019,68(2): 1079-1092.
[17]	GU B , ZHOU Z Y . Task offloading in vehicular mobile edge computing:a matching-theoretic framework[J]. IEEE Vehicular Technology Magazine, 2019,14(3): 100-106.
[18]	MA X T , ZHAO J H , LI Q P ,et al. Reinforcement learning based task offloading and take-back in vehicle platoon networks[C]// Proceedings of 2019 IEEE International Conference on Communications Workshops. Piscataway:IEEE Press, 2019: 1-6.
[19]	XU S Y , LIU Q C , GONG B ,et al. RJCC:reinforcement-learningbased joint communicational-and-computational resource allocation mechanism for smart city IoT[J]. IEEE Internet of Things Journal, 2020,7(9): 8059-8076.
[20]	KIRAN N , PAN C Y , WANG S H ,et al. Joint resource allocation and computation offloading in mobile edge computing for SDN based wireless networks[J]. Journal of Communications and Networks, 2020,22(1): 1-11.
[21]	JIANG F B , WANG K Z , DONG L ,et al. Deep-learning-based joint resource scheduling algorithms for hybrid MEC networks[J]. IEEE Internet of Things Journal, 2020,7(7): 6252-6265.
[22]	WANG R Y , JIANG X , ZHOU Y J ,et al. Multi-agent reinforcement learning for edge information sharing in vehicular networks[J]. Digital Communications and Networks, 2022,8(3): 267-277.
[23]	LI Y , WANG T S , WU Y ,et al. Optimal dynamic spectrum allocation-assisted latency minimization for multiuser mobile edge computing[J]. Digital Communications and Networks, 2022,8(3): 247-256.
[24]	YANG T T , FENG H L , GAO S ,et al. Two-stage offloading optimization for energy-latency tradeoff with mobile edge computing in maritime Internet of things[J]. IEEE Internet of Things Journal, 2020,7(7): 5954-5963.
[25]	苏新, 薛淏阳, 周一青 ,等. 面向海洋观监测传感网的计算卸载方法研究[J]. 通信学报, 2021,42(5): 149-163.
	SU X , XUE H Y , ZHOU Y Q ,et al. Research on computing offloading method for maritime observation monitoring sensor network[J]. Journal on Communications, 2021,42(5): 149-163.
[26]	苏新, 王子怡, 王宇鹏 ,等. 海洋观监测传感器网络多接入边缘计算卸载方法[J]. 物联网学报, 2021,5(1): 36-52.
	SU X , WANG Z Y , WANG Y P ,et al. Multi-access edge computing offloading in maritime monitoring sensor networks[J]. Chinese Journal on Internet of Things, 2021,5(1): 36-52.
[27]	TRAN T X , POMPILI D . Joint task offloading and resource allocation for multi-server mobile-edge computing networks[J]. IEEE Transactions on Vehicular Technology, 2019,68(1): 856-868.
[28]	MENG A Q , WEI G D , ZHAO Y ,et al. Green resource allocation for mobile edge computing[J]. Digital Communications and Networks, 2022:doi.org/10.1016/j.dcan.2022.03.001.
[29]	LIU L , ZHOU Y Q , GARCIA V ,et al. Load aware joint CoMP clustering and inter-cell resource scheduling in heterogeneous ultra dense cellular networks[J]. IEEE Transactions on Vehicular Technology, 2018,67(3): 2741-2755.
[30]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[31]	WANG H D , ZHOU J , LIU W ,et al. BGD-based Adam algorithm for time-domain equalizer in PAM-based optical interconnects[J]. Optics Letters, 2020,45(1): 141.
[32]	张鹏, 田辉, 赵鹏涛 ,等. 多智能体协作场景下基于强化学习值分解的计算卸载策略[J]. 通信学报, 2021,42(6): 1-15.
	ZHANG P , TIAN H , ZHAO P ,et al. Computation offloading strategy in multi-agent cooperation scenario based on reinforcement learning with value-decomposition[J]. Journal on Communications, 2021,42(6): 1-15.
[33]	ZHANG W W , WEN Y G , WU D O . Collaborative task execution in mobile cloud computing under a stochastic wireless channel[J]. IEEE Transactions on Wireless Communications, 2015,14(1): 81-93.

Metrics

Recommended 0

No Suggested Reading articles found!

文献	应用场景	系统模型	卸载算法	是否基于人工智能	是否考虑异构性	节点是否分层	实现效率	适用规模	是否存在维数灾难
文献[8]	蜂窝网	单服务器	Stackelberg博弈	否	否	否	低	小	是
文献[9]	蜂窝网	多服务器	逐次逼近	否	是	否	低	小	是
文献[10]	蜂窝网	单服务器	启发式	否	否	否	低	小	是
文献[11]	蜂窝网	单服务器	动态规划	否	否	否	较低	小	是
文献[12]	蜂窝网	多服务器	迭代算法	否	是	否	较低	小	是
文献[13]	车联网	多服务器	预测卸载	否	是	否	低	小	是
文献[14]	车联网	多服务器	启发式	否	是	否	较低	小	是
文献[15]	车联网	多服务器	移动感知	否	是	否	低	小	是
文献[16]	车联网	多服务器	Lyapunov优化	否	是	否	较低	小	是
文献[17]	车联网	多服务器	启发式	否	否	否	较低	小	是
文献[18]	车联网	单服务器	Q-Learning	是	否	否	较高	小	是
文献[19]	蜂窝网	多服务器	Q-Learning	是	否	否	较高	小	是
文献[20]	蜂窝网	多服务器	Q-Learning，SDN	是	是	否	较高	小	是
文献[21]	混合MEC网络	多服务器	DNN	是	是	否	较高	大	否
文献[22]	车联网	多服务器	DRL	是	是	否	高	大	否
文献[23]	蜂窝网	多服务器	DRL	是	是	否	高	大	否
文献[24]	海洋网络	单服务器	启发式	否	否	否	低	小	是
文献[25]	海洋网络	多服务器	粒子群优化、遗传算法	否	是	否	低	小	是
文献[26]	海洋网络	单服务器	人工鱼群算法	否	是	否	低	小	是

参数	数值
神经元层数/层	4
学习率	0.01
神经网络训练间隔	10
经验存储器容量/个	1 000
批规模/个	128
并行DNN数量/个	4

Maritime mobile edge computing offloading method based on deep reinforcement learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 33

Related Articles 15

Metrics

Recommended 0

[1]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[2]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[3]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[4]	Weifeng LU, Ning LI, Jia XU, Lijie XU, Jian XU. Joint scheduling algorithm for correlative tasks in multi-access edge computing [J]. Journal on Communications, 2023, 44(4): 87-98.
[5]	Xin SUN, Guifu ZHANG, Hongyan XING, Wang Zenghui. Research on intrusion detection for maritime meteorological sensor network based on balancing generative adversarial network [J]. Journal on Communications, 2023, 44(4): 124-136.
[6]	Renchao XIE, Wen WEN, Qinqin TANG, Yunlong LIU, Gaochang XIE, Tao HUANG. Survey on rail transit mobile edge computing network security [J]. Journal on Communications, 2023, 44(4): 201-215.
[7]	Xueyong YU, Lixiang QIU, Jianing SONG, Hongbo ZHU. Security communication and energy efficiency optimization strategy in UAV-aided edge computing [J]. Journal on Communications, 2023, 44(3): 45-54.
[8]	Wei FAN, Cheng PENG, Dali ZHU, Yuqing WANG. Research on intrusion response strategy based on static Bayesian game in mobile edge computing network [J]. Journal on Communications, 2023, 44(2): 70-81.
[9]	Shuxu ZHAO, Ping WEI, Xiaolong WANG. Optimal coalition structure generation strategy in multi-task concurrent edge computing environment [J]. Journal on Communications, 2023, 44(2): 172-184.
[10]	Long LONG, Zichen LIU, Zaiwang LU, Yucheng ZHANG, Lei LI. Joint optimization strategy of service cache and resource allocation in mobile edge network [J]. Journal on Communications, 2023, 44(1): 64-74.
[11]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[12]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[13]	Ziyuan WANG, Ruizhong DU. Certificateless public key cryptography based provable data possession scheme in edge environment [J]. Journal on Communications, 2022, 43(7): 62-72.
[14]	Zijia MO, Zhipeng GAO, Yang YANG, Yijing LIN, Shan SUN, Chen ZHAO. Efficient distributed model sharing strategy for data privacy protection in Internet of vehicles [J]. Journal on Communications, 2022, 43(4): 83-94.
[15]	Jifeng WANG, Guofeng WANG. Research on ciphertext search and sharing technology in edge computing mode [J]. Journal on Communications, 2022, 43(4): 227-238.