基于深度增强学习的无人机赋能雾无线电接入网络的能效优化

doi:10.11959/j.issn.2096-3750.2021.00234

Abstract

Abstract:

Fog radio access network (F-RAN) is suitable for Internet of things applications of national important industries, such as pipeline network monitoring in wide area.However, the performance of the F-RAN based on the territorial fog access point will be affected greatly by the complicated territorial environment.This causes F-RAN not able to provide fog access service in a timely and effectively manner.To this problem, the research was proposed to utilize low altitude UAV as the fog access point to realize air ground edge communication and fog computing, which has attracted enormous research interests.How to use deep reinforcement learning (DRL) to improve the energy efficiency of UAV fog access point and extend the mission time of UAV were discussed.Deep reinforcement learning can ensure the UAV fog access point to adjust the configuration strategy timely of air ground communication and computing, including resource optimization, dynamic task offloading and caching.DRL can also optimize the UAV trajectory in 3-D space, and improve the overall performance of UAV enabled fog access network.The innovation of the research lies in the comprehensive discussion of the main optimization problems to be solved in the UAV-enabled F-RAN using DRL.The technical details were also summarized to solve the related optimization problems.Finally, the technical challenges and future research directions of the application of DRL in the UAV-enabled F-RAN were discussed.

Key words: unmanned aerial vehicle, fog radio access network, deep reinforcement learning, trajectory design, network configuration

CLC Number:

TP393

Haibo MEI, Kun YANG, Xinyu FAN. Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN[J]. Chinese Journal on Internet of Things, 2021, 5(2): 48-59.

Figures/Tables 8

References 19

[1]	CHIANG M , ZHANG T . Fog and IoT:an overview of research opportunities[J]. IEEE Internet of Things Journal, 2016,3(6): 854-864.
[2]	LOKE S W . The Internet of flying-things:opportunities and challenges with airborne fog computing and mobile cloud in the clouds[J]. arXiv preprint arXiv:1507.04492, 2015.
[3]	ZENG Y , ZHANG R , LIM T J . Wireless communications with unmanned aerial vehicles:opportunities and challenges[J]. IEEE Communications Magazine, 2016,54(5): 36-42.
[4]	WU Q Q , LIU L , ZHANG R . Fundamental trade-offs in communication and trajectory design for UAV-enabled wireless network[J]. IEEE Wireless Communications, 2019,26(1): 36-44.
[5]	KU Y J , LIN D Y , LEE C F ,et al. 5G radio access network design with the fog paradigm:confluence of communications and computing[J]. IEEE Communications Magazine, 2017,55(4): 46-52.
[6]	ZENG Y , XU J , ZHANG R . Energy minimization for wireless communication with rotary-wing UAV[J]. IEEE Transactions on Wireless Communications, 2019,18(4): 2329-2345.
[7]	ZENG Y , ZHANG R . Energy-efficient UAV communication with trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2017,16(6): 3747-3760.
[8]	MEI H B , WANG K Z , ZHOU D D ,et al. Joint trajectory-task-cache optimization in UAV-enabled mobile edge networks for cyber-physical system[J]. IEEE Access, 2019(7): 156476-156488.
[9]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[10]	LI R P , ZHAO Z F , ZHOU X ,et al. Intelligent 5G:when cellular networks meet artificial intelligence[J]. IEEE Wireless Communications, 2017,24(5): 175-183.
[11]	CHEN M Z , MOZAFFARI M , SAAD W ,et al. Caching in the sky:proactive deployment of cache-enabled unmanned aerial vehicles for optimized quality-of-experience[J]. IEEE Journal on Selected Areas in Communications, 2017,35(5): 1046-1061.
[12]	CHEN M Z , SAAD W , YIN C C . Liquid state machine learning for resource and cache management in LTE-U unmanned aerial vehicle (UAV) networks[J]. IEEE Transactions on Wireless Communications, 2019,18(3): 1504-1517.
[13]	BOYD S , VANDENBERGHE L . Convex optimization[M]. Cambridge: Cambridge University Press, 2004.
[14]	MEI H B , YANG K , LIU Q ,et al. Joint trajectory-resource optimization in UAV-enabled edge-cloud system with virtualized mobile clone[J]. IEEE Internet of Things Journal, 2020,7(7): 5906-5921.
[15]	HE Y , YU F R , ZHAO N ,et al. Software-defined networks with mobile edge computing and caching for smart cities:a big data deep reinforcement learning approach[J]. IEEE Communications Magazine, 2017,55(12): 31-37.
[16]	CHALLITA U , SAAD W , BETTSTETTER C . Deep reinforcement learning for interference-aware path planning of cellular-connected UAVs[C]// 2018 IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2018: 1-7.
[17]	LIU C H , CHEN Z Y , TANG J ,et al. Energy-efficient UAV control for effective and fair communication coverage:a deep reinforcement learning approach[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2059-2070.
[18]	MEI H B , YANG K , SHEN J ,et al. Joint trajectory-task-cache optimization with phase-shift design of RIS-assisted UAV for MEC[EB]. IEEE Wireless Communications Letters, 2021.
[19]	PENG H X , SHEN X S . DDPG-based resource management for MEC/UAV-assisted vehicular networks[C]// 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall). Piscataway:IEEE Press, 2020: 1-6.

Metrics

Recommended 0

No Suggested Reading articles found!

Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 19

Related Articles 11

Metrics

Recommended 0

[1]	Cenhuishan LIAO, Junyan CHEN, Guanping LIANG, Xiaolan XIE, Xiaoye LU. Quality of service optimization algorithm based on deep reinforcement learning in software defined network [J]. Chinese Journal on Internet of Things, 2023, 7(1): 73-82.
[2]	Hanqing YU, Yan LIN, Linqiong JIA, Qiang LI, Yijin Zhang. A distributed strategy for the multi-target rescue using a UAV swarm under communication constraints [J]. Chinese Journal on Internet of Things, 2022, 6(3): 103-112.
[3]	Qianwen LI, Jianfeng CHEN, Miao CUI, Guangchi ZHANG. Trajectory and communication scheduling optimization for the rechargeable UAV aided data collection system [J]. Chinese Journal on Internet of Things, 2022, 6(3): 113-123.
[4]	Zihui LUO, Chengling JIANG, Liang LIU, Xiaolong ZHENG, Huadong MA. Research on deep reinforcement learning based intelligent shop scheduling method [J]. Chinese Journal on Internet of Things, 2022, 6(1): 53-64.
[5]	Wei WANG, Yajing LIANG, Li PENG, Zhongcheng WEI, Jijun ZHAO. Node clustered deployment of emergency Internet of things based on UAV with equipment access restriction [J]. Chinese Journal on Internet of Things, 2021, 5(3): 97-105.
[6]	Jiequ JI, Kun ZHU, Changyan YI, Ran WANG. Joint task offloading and trajectory optimization for multi-UAV assisted mobile edge computing [J]. Chinese Journal on Internet of Things, 2021, 5(1): 27-35.
[7]	Zhiyu MOU,Yu ZHANG,Dian FAN,Jun LIU,Feifei GAO. Research on the UAV-aided data collection and trajectory design based on the deep reinforcement learning [J]. Chinese Journal on Internet of Things, 2020, 4(3): 42-51.
[8]	Yalin LIU,Hongning DAI,Qubeijian WANG. Unmanned aerial vehicle enabled communication technologies and applications for Internet of things [J]. Chinese Journal on Internet of Things, 2019, 3(4): 48-55.
[9]	Yi ZHOU,Xiaoyong MA,Fuxiao GAO,Wei LI,Nan CHENG,Ning LU. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning [J]. Chinese Journal on Internet of Things, 2019, 3(2): 47-55.
[10]	Ruijin DING,Feifei GAO,Ling XING. Intelligent routing strategy in the Internet of things based on deep reinforcement learning [J]. Chinese Journal on Internet of Things, 2019, 3(2): 56-63.
[11]	Guanhua QIAO,Supeng LENG,Hao LIU,Kaisheng HUANG,Fan WU. Task collaborative offloading scheme in vehicle multi-access edge computing network [J]. Chinese Journal on Internet of Things, 2019, 3(1): 51-59.