面向多目标救援的通信受限无人机集群分布式策略

doi:10.11959/j.issn.2096-3750.2022.00284

物联网学报 ›› 2022, Vol. 6 ›› Issue (3): 103-112.doi: 10.11959/j.issn.2096-3750.2022.00284

面向多目标救援的通信受限无人机集群分布式策略

俞汉清¹, 林艳¹^,², 贾林琼¹, 李强³, 张一晋¹

¹ 南京理工大学电子工程与光电技术学院，江苏南京 210094
² 东南大学移动通信国家重点实验室，江苏南京 210096
³ 鹏城实验室，广东深圳 518000

修回日期:2022-06-15 出版日期:2022-08-05 发布日期:2022-08-08
作者简介:俞汉清（1999- ），男，南京理工大学电子工程与光电技术学院在读，CCF 学生会员，主要研究方向为深度学习、强化学习在无线网络的应用
林艳（1990- ），女，博士，南京理工大学副教授，主要研究方向为6G无线资源分配、强化学习等
贾林琼（1989- ），女，博士，南京理工大学讲师，主要研究方向为可见光通信与移动通信
李强（1973- ），男，博士，鹏城实验室正高级工程师，主要研究方向为物联网与5G/B5G
张一晋（1982- ），博士，南京理工大学教授，主要研究方向为序列设计、无线网络与人工智能
基金资助:
国家自然科学基金资助项目(62071236);国家自然科学基金资助项目(62001225);中央高校基本科研业务费资助项目(30920021127);江苏省自然科学基金资助项目(BK20190454);鹏城实验室重大攻关项目(PCL2021A15);东南大学移动通信国家重点实验室开放研究基金资助项目(2022D07)

A distributed strategy for the multi-target rescue using a UAV swarm under communication constraints

Hanqing YU¹, Yan LIN¹^,², Linqiong JIA¹, Qiang LI³, Yijin Zhang¹

¹ School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
² National Mobile Communications Research Laboratory, Southeast University, Nanjing 210096, China
³ Peng Cheng Laboratory, Shenzhen 518000, China

Revised:2022-06-15 Online:2022-08-05 Published:2022-08-08
Supported by:
The National Natural Science Foundation of China(62071236);The National Natural Science Foundation of China(62001225);The Fundamental Research Funds for the Central Universities of China(30920021127);The Natural Science Foundation of Jiangsu Province(BK20190454);The Major Key Project of PCL(PCL2021A15);The Open Research Fund of National Mobile Communications Research Laboratory, Southeast University(2022D07)

摘要/Abstract

摘要：

现有无人机集群的协同决策设计所依据的信息共享缺乏对无人机之间通信能力的合理假设。针对电量、载荷和路线约束下的无人机集群多目标救援问题，结合无人机飞行路线，考虑通信能力对无人机之间信息共享的限制。首先，将问题建模成部分可观测马尔可夫决策过程；然后，利用循环神经网络提出基于深度强化学习的能够适应通信拓扑结构不断变化的分布式救援策略。仿真结果表明，所提策略相较于其他策略在通信受限的情况下具有更佳的分布式救援性能，无人机数量和无人机通信能力需要依据救援场景进行联合设置方能达到无人机集群救援性能和使用成本的最佳折中。

关键词: 无人机, 多目标救援, 马尔可夫决策过程, 分布式策略, 强化学习

Abstract:

The current designs of the cooperative decision-making of an unmanned aerial vehicle (UAV) swarm usually adopt unreasonable assumptions on the communication ability between UAVs.Focusing on a multi-target rescue problem of a UAV swarm under constraints of energy, load and path, the limitation on the information sharing due to the communication constraints and the flight path of UAVs were taken into account.Firstly, the problem was formulated as a partially observable Markov decision process (POMDP).Then, a recurrent neural network was used to propose a deep-reinforcement-learning-based distributed rescue strategy, which is able to adapt to the changeable communication topology.Simulation results show that the proposed strategy outperforms other strategies under communication constraints, and further show that a careful joint setting of the size and communication ability of a UAV swarm is needed to achieve the best compromise between the UAV swarm rescue performance and the cost.

Key words: unmanned aerial vehicle, multi-target rescue, Markov decision process, distributed strategy, reinforcement learning

中图分类号:

TN911

俞汉清, 林艳, 贾林琼, 李强, 张一晋. 面向多目标救援的通信受限无人机集群分布式策略[J]. 物联网学报, 2022, 6(3): 103-112.

Hanqing YU, Yan LIN, Linqiong JIA, Qiang LI, Yijin Zhang. A distributed strategy for the multi-target rescue using a UAV swarm under communication constraints[J]. Chinese Journal on Internet of Things, 2022, 6(3): 103-112.

图/表 10

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

参考文献 30

[1]	韩亮, 任章, 董希旺 ,等. 多无人机协同控制方法及应用研究[J]. 导航定位与授时, 2018,5(4): 1-7.
	HAN L , REN Z , DONG X W ,et al. Research on cooperative control method and application for multiple unmanned aerial vehicles[J]. Navigation Positioning and Timing, 2018,5(4): 1-7.
[2]	HILDMANN H , KOVACS E . Review,using unmanned aerial vehicles (UAVs) as mobile sensing platforms (MSPs) for disaster response,civil security and public safety[J]. Drones, 2019,3(3): 59.
[3]	FAN B K , LI Y , ZHANG R Y ,et al. Review on the technological development and application of UAV systems[J]. Chinese Journal of Electronics, 2020,29(2): 199-207.
[4]	CAMPION M , RANGANATHAN P , FARUQUE S . UAV swarm communication and control architectures,a review[J]. Journal of Unmanned Vehicle Systems, 2019,7(2): 93-106.
[5]	ZHOU Y K , RAO B , WANG W . UAV swarm intelligence,recent advances and future trends[J]. IEEE Access, 8: 183856-183878.
[6]	BACCO M , CHESSA S , BENEDETTO M ,et al. UAVs and UAV swarms for civilian applications,communications and image processing in the SCIADRO project[C]// Wireless and Satellite Systems, 2018: 115-124.
[7]	RUETTEN L , REGIS P A , FEIL-SEIFER D ,et al. Area-optimized UAV swarm network for search and rescue operations[C]// Proceedings of 2020 10th Annual Computing and Communication Workshop and Conference (CCWC). Piscataway,IEEE Press, 2020: 613-618.
[8]	MATESE A , TOSCANO P , DI GENNARO S ,et al. Intercomparison of UAV,aircraft and satellite remote sensing platforms for precision viticulture[J]. Remote Sensing, 2015,7(3): 2971-2990.
[9]	KIM K S , KIM H Y , CHOI H L . A bid-based grouping method for communication-efficient decentralized multi-UAV task allocation[J]. International Journal of Aeronautical and Space Sciences, 2020,21(1): 290-302.
[10]	LADOSZ P , OH H , ZHENG G ,et al. Gaussian process based channel prediction for communication-relay UAV in urban environments[J]. IEEE Transactions on Aerospace and Electronic Systems, 2020,56(1): 313-325.
[11]	宗群, 王丹丹, 邵士凯 ,等. 多无人机协同编队飞行控制研究现状及发展[J]. 哈尔滨工业大学学报, 2017,49(3): 1-14.
	ZONG Q , WANG D D , SHAO S K ,et al. Research status and development of multi UAV coordinated formation flight control[J]. Journal of Harbin Institute of Technology, 2017,49(3): 1-14.
[12]	张可为, 赵晓林, 李宗哲 ,等. 多无人机侦察任务分配方法研究综述[J]. 电光与控制, 2021,28(7): 68-72,82.
	ZHANG K W , ZHAO X L , LI Z Z ,et al. A review of multi-UAV reconnaissance mission assignment methods[J]. Electronics Optics ＆Control, 2021,28(7): 68-72,82.
[13]	陈少飞 . 无人机集群系统侦察监视任务规划方法[D]. 长沙,国防科学技术大学, 2016.
	CHEN S F . Planning for reconnaissance and monitoring using UAV swarms[D]. Changsha,National University of Defense Technology, 2016.
[14]	ZHAO J W , ZHAO J J . Study on multi-UAV task clustering and task planning in cooperative reconnaissance[C]// Proceedings of 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics. Piscataway,IEEE Press, 2014: 392-395.
[15]	黄捷, 陈谋, 姜长生 . 无人机空对地多目标攻击的满意分配决策技术[J]. 电光与控制, 2014,21(7): 10-13,30.
	HUANG J , CHEN M , JIANG C S . Satisficing decision-making on task allocation for UAVs in air-to-ground attacking[J]. Electronics Optics ＆ Control, 2014,21(7): 10-13,30.
[16]	SU C X , YE F , WANG L C ,et al. UAV-assisted wireless charging for energy-constrained IoT devices using dynamic matching[J]. IEEE Internet of Things Journal, 2020,7(6): 4789-4800.
[17]	ABEDIN S F , MUNIR M S , TRAN N H ,et al. Data freshness and energy-efficient UAV navigation optimization,a deep reinforcement learning approach[J]. IEEE Transactions on Intelligent Transportation Systems, 2021,22(9): 5994-6006.
[18]	WHITBROOK A , MENG Q G , CHUNG P W H . Reliable,distributed scheduling and rescheduling for time-critical,multiagent systems[J]. IEEE Transactions on Automation Science and Engineering, 2018,15(2): 732-747.
[19]	杜永浩, 邢立宁, 蔡昭权 . 无人飞行器集群智能调度技术综述[J]. 自动化学报, 2020,46(2): 222-241.
	DU Y H , XING L N , CAI Z Q . Survey on intelligent scheduling technologies for unmanned flying craft clusters[J]. Acta Automatica Sinica, 2020,46(2): 222-241.
[20]	BALDAZO D , PARRAS J , ZAZO S . Decentralized multi-agent deep reinforcement learning in swarms of drones for flood monitoring[C]// Proceedings of 2019 27th European Signal Processing Conference (EUSIPCO). Piscataway,IEEE Press, 2019: 1-5.
[21]	左益宏, 柳长安, 罗昌行 ,等. 多无人机监控航路规划[J]. 飞行力学, 2004,22(3): 31-34.
	ZUO Y H , LIU C G , LUO C X ,et al. Path planning for surveillance of multiple unmanned air vehicles[J]. Flight Dynamics, 2004,22(3): 31-34.
[22]	WU H S , LI H , XIAO R B ,et al. Modeling and simulation of dynamic ant colony's labor division for task allocation of UAV swarm[J]. Physica A,Statistical Mechanics and Its Applications, 2018,491: 127-141.
[23]	成成, 张跃, 储海荣 ,等. 分布式多无人机协同编队队形控制仿真[J]. 计算机仿真, 2019,36(5): 31-37.
	CHENG C , ZHANG Y , CHU H R ,et al. Simulation of distributed cooperative formation control for multi-UAVs[J]. Computer Simulation, 2019,36(5): 31-37.
[24]	ZHAO Y Y , WANG X K , WANG C ,et al. Systemic design of distributed multi-UAV cooperative decision-making for multi-target tracking[J]. Autonomous Agents and Multi-Agent Systems, 2019,33(1/2): 132-158.
[25]	FU X W , PAN J , WANG H X ,et al. A formation maintenance and reconstruction method of UAV swarm based on distributed control[J]. Aerospace Science and Technology, 2020,104,105981.
[26]	SAMVELYAN M , RASHID T , DE WITT C S ,et al. The StarCraft multi-agent challenge[C]// Proceedings of AAMAS '19,Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems. 2019: 2186-2188.
[27]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[28]	KRAEMER L , BANERJEE B . Multi-agent reinforcement learning as a rehearsal for decentralized planning[J]. Neurocomputing, 2016,190: 82-94.
[29]	WANG Z , SCHAUL T , HESSEL M ,et al. Dueling network architectures for deep reinforcement learning[C]// International Conference on Machine Learning. PMLR, 2016: 1995-2003.
[30]	TAMPUU A , MATIISEN T , KODELJA D ,et al. Multiagent cooperation and competition with deep reinforcement learning[J]. PLoS One, 2017,12(4): e0172395.

面向多目标救援的通信受限无人机集群分布式策略

A distributed strategy for the multi-target rescue using a UAV swarm under communication constraints

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 30

相关文章 15

Metrics

推荐阅读 0

[1]	王志宏, 冷甦鹏, 熊凯. 面向无人机集群协同感知的多智能体资源分配策略[J]. 物联网学报, 2023, 7(1): 18-26.
[2]	廖岑卉珊, 陈俊彦, 梁观平, 谢小兰, 卢小烨. 基于深度强化学习的SDN服务质量智能优化算法[J]. 物联网学报, 2023, 7(1): 73-82.
[3]	张彪, 汪西明, 徐逸凡, 李文, 韩昊, 刘松仪, 陈学强. 基于多智能体深度强化学习的多域协同抗干扰方法研究[J]. 物联网学报, 2022, 6(4): 104-116.
[4]	张欢欢, 周安福, 马华东. 基于强化学习的实时视频流控与移动终端训练方法研究[J]. 物联网学报, 2022, 6(4): 1-13.
[5]	杨澳钦, 宫傲宇, 房婷, 邓磊, 李强, 张一晋. 传输时限约束下的能量收集无线传感器网络多址接入优化[J]. 物联网学报, 2022, 6(3): 58-70.
[6]	陈九九, 郭彩丽, 冯春燕, 刘传宏. 智能网联环境下面向语义通信的资源分配[J]. 物联网学报, 2022, 6(3): 47-57.
[7]	李茜雯, 陈健锋, 崔苗, 张广驰. 可充电无人机辅助数据采集系统的飞行路线与通信调度优化[J]. 物联网学报, 2022, 6(3): 113-123.
[8]	罗梓珲, 江呈羚, 刘亮, 郑霄龙, 马华东. 基于深度强化学习的智能车间调度方法研究[J]. 物联网学报, 2022, 6(1): 53-64.
[9]	王巍, 谷壬倩, 彭力, 赵继军, 魏忠诚, 常存喜. 基于无人机的物联网空基中继鲁棒优化[J]. 物联网学报, 2022, 6(1): 101-112.
[10]	王巍, 梁雅静, 彭力, 魏忠诚, 赵继军. 设备接入受限的UAV空基应急物联网节点分簇部署研究[J]. 物联网学报, 2021, 5(3): 97-105.
[11]	梅海波, 杨鲲, 范新宇. 基于深度增强学习的无人机赋能雾无线电接入网络的能效优化[J]. 物联网学报, 2021, 5(2): 48-59.
[12]	林椿珉, 曾烈康, 陈旭. 边缘智能驱动的高能效无人机自主导航算法研究[J]. 物联网学报, 2021, 5(2): 87-96.
[13]	嵇介曲, 朱琨, 易畅言, 王然. 多无人机辅助移动边缘计算中的任务卸载和轨迹优化[J]. 物联网学报, 2021, 5(1): 27-35.
[14]	沈学民,承楠,周海波,吕丰,权伟,时伟森,吴华清,周淙浩. 空天地一体化网络技术：探索与展望[J]. 物联网学报, 2020, 4(3): 3-19.
[15]	牟治宇,张煜,范典,刘君,高飞飞. 基于深度强化学习的无人机数据采集和路径规划研究[J]. 物联网学报, 2020, 4(3): 42-51.