基于深度增强学习的无人机赋能雾无线电接入网络的能效优化

doi:10.11959/j.issn.2096-3750.2021.00234

物联网学报 ›› 2021, Vol. 5 ›› Issue (2): 48-59.doi: 10.11959/j.issn.2096-3750.2021.00234

• 专题：物联网边缘智能与雾计算技术 • 上一篇下一篇

基于深度增强学习的无人机赋能雾无线电接入网络的能效优化

梅海波, 杨鲲, 范新宇

电子科技大学，四川成都 610054

修回日期:2021-04-29 出版日期:2021-06-30 发布日期:2021-06-01
作者简介:梅海波（1983- ），男，博士，电子科技大学讲师，主要研究方向为无线移动通信、移动边缘计算、无人机通信、人工智能
杨鲲（1969- ），男，电子科技大学教授，主要研究方向为无线通信网络、数能一体化通信网、移动计算
范新宇（1997- ），男，电子科技大学硕士生，主要研究方向为物联网、数能一体化通信网
基金资助:
国家自然科学基金资助项目(61620106011);国家自然科学基金资助项目(U1705263);国家自然科学基金资助项目(61871076)

Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN

Haibo MEI, Kun YANG, Xinyu FAN

University of Electronic Science and Technology of China, Chengdu 610054, China

Revised:2021-04-29 Online:2021-06-30 Published:2021-06-01
Supported by:
The National Natural Science Foundation of China(61620106011);The National Natural Science Foundation of China(U1705263);The National Natural Science Foundation of China(61871076)

摘要/Abstract

摘要：

雾无线电接入网络适合用于广域范围内的诸如管线管网监测等国家重要行业的物联网应用场景。然而基于地面雾接入节点的网络将受到环境、地形等影响，无法及时有效地提供雾接入服务。利用低空无人机作为雾接入点实现空地的边缘通信和雾计算方面引起了普遍的关注。本文探讨怎样利用深度增强学习来提高无人机雾接入点的能效，延长无人机的任务时间。深度增强学习可以保障无人机雾接入点及时地调整空地通信和计算的配置策略，包括资源优化、动态任务卸载以及缓存，也可以优化无人机在三维空间中的飞行航迹，提高无人机赋能的雾无线电接入网络的总体性能。研究的创新性在于综合论述了深度增强学习用于无人机赋能的雾无线电接入网络要解决的主要优化问题，并且总结了解决相关优化问题的技术细节，最后对深度增强学习应用于无人机赋能的雾无线电接入网络的技术挑战和未来研究方向展开讨论。

关键词: 无人机, 雾无线电接入网络, 深度增强学习, 航迹规划, 网络配置

Abstract:

Fog radio access network (F-RAN) is suitable for Internet of things applications of national important industries, such as pipeline network monitoring in wide area.However, the performance of the F-RAN based on the territorial fog access point will be affected greatly by the complicated territorial environment.This causes F-RAN not able to provide fog access service in a timely and effectively manner.To this problem, the research was proposed to utilize low altitude UAV as the fog access point to realize air ground edge communication and fog computing, which has attracted enormous research interests.How to use deep reinforcement learning (DRL) to improve the energy efficiency of UAV fog access point and extend the mission time of UAV were discussed.Deep reinforcement learning can ensure the UAV fog access point to adjust the configuration strategy timely of air ground communication and computing, including resource optimization, dynamic task offloading and caching.DRL can also optimize the UAV trajectory in 3-D space, and improve the overall performance of UAV enabled fog access network.The innovation of the research lies in the comprehensive discussion of the main optimization problems to be solved in the UAV-enabled F-RAN using DRL.The technical details were also summarized to solve the related optimization problems.Finally, the technical challenges and future research directions of the application of DRL in the UAV-enabled F-RAN were discussed.

Key words: unmanned aerial vehicle, fog radio access network, deep reinforcement learning, trajectory design, network configuration

中图分类号:

TP393

梅海波, 杨鲲, 范新宇. 基于深度增强学习的无人机赋能雾无线电接入网络的能效优化[J]. 物联网学报, 2021, 5(2): 48-59.

Haibo MEI, Kun YANG, Xinyu FAN. Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN[J]. Chinese Journal on Internet of Things, 2021, 5(2): 48-59.

图/表 8

图1

图2

图3

图4

图5

图6

图7

图8

参考文献 19

[1]	CHIANG M , ZHANG T . Fog and IoT:an overview of research opportunities[J]. IEEE Internet of Things Journal, 2016,3(6): 854-864.
[2]	LOKE S W . The Internet of flying-things:opportunities and challenges with airborne fog computing and mobile cloud in the clouds[J]. arXiv preprint arXiv:1507.04492, 2015.
[3]	ZENG Y , ZHANG R , LIM T J . Wireless communications with unmanned aerial vehicles:opportunities and challenges[J]. IEEE Communications Magazine, 2016,54(5): 36-42.
[4]	WU Q Q , LIU L , ZHANG R . Fundamental trade-offs in communication and trajectory design for UAV-enabled wireless network[J]. IEEE Wireless Communications, 2019,26(1): 36-44.
[5]	KU Y J , LIN D Y , LEE C F ,et al. 5G radio access network design with the fog paradigm:confluence of communications and computing[J]. IEEE Communications Magazine, 2017,55(4): 46-52.
[6]	ZENG Y , XU J , ZHANG R . Energy minimization for wireless communication with rotary-wing UAV[J]. IEEE Transactions on Wireless Communications, 2019,18(4): 2329-2345.
[7]	ZENG Y , ZHANG R . Energy-efficient UAV communication with trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2017,16(6): 3747-3760.
[8]	MEI H B , WANG K Z , ZHOU D D ,et al. Joint trajectory-task-cache optimization in UAV-enabled mobile edge networks for cyber-physical system[J]. IEEE Access, 2019(7): 156476-156488.
[9]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[10]	LI R P , ZHAO Z F , ZHOU X ,et al. Intelligent 5G:when cellular networks meet artificial intelligence[J]. IEEE Wireless Communications, 2017,24(5): 175-183.
[11]	CHEN M Z , MOZAFFARI M , SAAD W ,et al. Caching in the sky:proactive deployment of cache-enabled unmanned aerial vehicles for optimized quality-of-experience[J]. IEEE Journal on Selected Areas in Communications, 2017,35(5): 1046-1061.
[12]	CHEN M Z , SAAD W , YIN C C . Liquid state machine learning for resource and cache management in LTE-U unmanned aerial vehicle (UAV) networks[J]. IEEE Transactions on Wireless Communications, 2019,18(3): 1504-1517.
[13]	BOYD S , VANDENBERGHE L . Convex optimization[M]. Cambridge: Cambridge University Press, 2004.
[14]	MEI H B , YANG K , LIU Q ,et al. Joint trajectory-resource optimization in UAV-enabled edge-cloud system with virtualized mobile clone[J]. IEEE Internet of Things Journal, 2020,7(7): 5906-5921.
[15]	HE Y , YU F R , ZHAO N ,et al. Software-defined networks with mobile edge computing and caching for smart cities:a big data deep reinforcement learning approach[J]. IEEE Communications Magazine, 2017,55(12): 31-37.
[16]	CHALLITA U , SAAD W , BETTSTETTER C . Deep reinforcement learning for interference-aware path planning of cellular-connected UAVs[C]// 2018 IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2018: 1-7.
[17]	LIU C H , CHEN Z Y , TANG J ,et al. Energy-efficient UAV control for effective and fair communication coverage:a deep reinforcement learning approach[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2059-2070.
[18]	MEI H B , YANG K , SHEN J ,et al. Joint trajectory-task-cache optimization with phase-shift design of RIS-assisted UAV for MEC[EB]. IEEE Wireless Communications Letters, 2021.
[19]	PENG H X , SHEN X S . DDPG-based resource management for MEC/UAV-assisted vehicular networks[C]// 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall). Piscataway:IEEE Press, 2020: 1-6.

基于深度增强学习的无人机赋能雾无线电接入网络的能效优化

Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 19

相关文章 13

Metrics

推荐阅读 0

[1]	王志宏, 冷甦鹏, 熊凯. 面向无人机集群协同感知的多智能体资源分配策略[J]. 物联网学报, 2023, 7(1): 18-26.
[2]	俞汉清, 林艳, 贾林琼, 李强, 张一晋. 面向多目标救援的通信受限无人机集群分布式策略[J]. 物联网学报, 2022, 6(3): 103-112.
[3]	李茜雯, 陈健锋, 崔苗, 张广驰. 可充电无人机辅助数据采集系统的飞行路线与通信调度优化[J]. 物联网学报, 2022, 6(3): 113-123.
[4]	王巍, 谷壬倩, 彭力, 赵继军, 魏忠诚, 常存喜. 基于无人机的物联网空基中继鲁棒优化[J]. 物联网学报, 2022, 6(1): 101-112.
[5]	王巍, 梁雅静, 彭力, 魏忠诚, 赵继军. 设备接入受限的UAV空基应急物联网节点分簇部署研究[J]. 物联网学报, 2021, 5(3): 97-105.
[6]	林椿珉, 曾烈康, 陈旭. 边缘智能驱动的高能效无人机自主导航算法研究[J]. 物联网学报, 2021, 5(2): 87-96.
[7]	嵇介曲, 朱琨, 易畅言, 王然. 多无人机辅助移动边缘计算中的任务卸载和轨迹优化[J]. 物联网学报, 2021, 5(1): 27-35.
[8]	牟治宇,张煜,范典,刘君,高飞飞. 基于深度强化学习的无人机数据采集和路径规划研究[J]. 物联网学报, 2020, 4(3): 42-51.
[9]	刘亚林,戴弘宁,王曲北剑. 无人机辅助的物联网通信技术及其应用[J]. 物联网学报, 2019, 3(4): 48-55.
[10]	周毅,马晓勇,郜富晓,李伟,承楠,路宁. 基于深度强化学习的无人机自主部署及能效优化策略[J]. 物联网学报, 2019, 3(2): 47-55.
[11]	乔冠华,冷甦鹏,刘浩,黄开胜,吴凡. 面向车辆多址接入边缘计算网络的任务协同计算迁移策略[J]. 物联网学报, 2019, 3(1): 51-59.
[12]	宋庆恒,郑福春. 基于无人机的物联网无线通信的潜力与方法[J]. 物联网学报, 2019, 3(1): 82-89.
[13]	郭芳,张玉艳,赵龙,郑侃,王文博. 基于无人机辅助的V2I无线传输系统性能优化[J]. 物联网学报, 2017, 1(2): 46-53.