有人机/无人机智能协同目标搜索和轨迹规划算法

doi:10.11959/j.issn.1000-436x.2024003

通信学报 ›› 2024, Vol. 45 ›› Issue (1): 31-40.doi: 10.11959/j.issn.1000-436x.2024003

• 专题：面向有人无人协同的智能通信与组网技术 • 上一篇

有人机/无人机智能协同目标搜索和轨迹规划算法

卢卓, 吴启晖, 周福辉

南京航空航天大学电子信息工程学院，江苏南京 210016

修回日期:2023-09-20 出版日期:2024-01-01 发布日期:2024-01-01
作者简介:卢卓（1994- ），女，湖南长沙人，南京航空航天大学博士生，主要研究方向为无人机群智能协同通信、多智能体强化学习、无人机轨迹规划等
吴启晖（1970- ），男，安徽歙县人，博士，南京航空航天大学特聘教授，主要研究方向为认知信息论、电磁空间频谱智能管控、天地一体化信息网络、无人机集群智能通信等
周福辉（1988- ），男，江西抚州人，博士，南京航空航天大学教授，主要研究方向为认知智能、频谱智能管控、资源智能优化、语义通信等
基金资助:
江苏省基础研究计划自然科学基金资助项目(BK20222013);江苏省科研与实践创新计划基金资助项目(KYCX22_0358)

Algorithm for intelligent collaborative target search and trajectory planning of MAV/UAV

Zhuo LU, Qihui WU, Fuhui ZHOU

College of Electronic and Information Engineering, Nanjing University of Aeronautics ＆Astronautics, Nanjing 210016, China

Revised:2023-09-20 Online:2024-01-01 Published:2024-01-01
Supported by:
Basic Research Program Natural Science Foundation of Jiangsu Province(BK20222013);The Postgraduate Research and Practice Innovation Program of Jiangsu Province(KYCX22_0358)

摘要/Abstract

摘要：

基于有人机/无人机智能协同平台，针对多个位置未知的干扰信号源搜索及轨迹规划进行了研究。考虑到搜索过程的实时性和动态性，提出了一种基于多智能体深度强化学习的有人机/无人机智能协同目标搜索和轨迹规划（MUICTSTP）算法。各无人机通过感知接收干扰信号强度在线决策轨迹规划，同时将感知信息和决策动作传给有人机来获得全局评估。仿真结果表明，该算法相比其他算法在长期接收干扰信号强度、碰撞等方面表现出更好性能，且获得更优的学习策略。

关键词: 有人机/无人机, 智能协同, 多智能体深度强化学习, 轨迹规划, 接收干扰信号强度

Abstract:

Based on the manned aerial vehicle (MAV) / unmanned aerial vehicle (UAV) intelligent cooperation platform, the search of multiple interfered signal sources with unknown locations and trajectory planning were studied.Considering the real-time and dynamic nature of the search process, a MAV/UAV intelligent collaborative target search and trajectory planning (MUICTSTP) algorithm based on multi-agent deep reinforcement learning (MADRL) was proposed.Each UAV made online decision on trajectory planning by sensing the received interference signal strength (RISS) values, and then transmitted the sensing information and decision-making actions to the MAV to obtain the global evaluation.The simulation results show that the proposed algorithm exhibits better performance in long-term RISS, collision, and other aspects compared to other algorithms, and the learning strategy is better.

Key words: MAV/UAV, intelligent collaborative, MADRL, trajectory planning, RISS

中图分类号:

TN92

卢卓, 吴启晖, 周福辉. 有人机/无人机智能协同目标搜索和轨迹规划算法[J]. 通信学报, 2024, 45(1): 31-40.

Zhuo LU, Qihui WU, Fuhui ZHOU. Algorithm for intelligent collaborative target search and trajectory planning of MAV/UAV[J]. Journal on Communications, 2024, 45(1): 31-40.

图/表 10

图1

表1

系统参数及其含义"

参数	含义
M ,U,N	有人机数量、无人机数量、干扰信号源数量
d_ij(t)	无人机i与干扰信号源 j在时刻t的距离
P_ij(t)	无人机i与干扰信号源 j在时刻t的瞬时功率
L,G ^T,G^R	路径损耗、发射天线增益、接收天线增益
$θ_{i}^{π}, θ_{m, i}^{Q}$	actor网络权重、critic网络权重
$θ_{i}^{π}^{'}, θ_{m, i}^{Q}^{'}$	target actor网络权重、target critic网络权重
B	经验回放池大小
N_b	样本大小
φ	样本索引
τ	软更新率
γ	折扣因子

表1

图2

图3

图4

图5

图6

图7

图8

图9

参考文献 21

[1]	姚富强, 张余, 柳永祥 . 电磁频谱安全与控制[J]. 指挥与控制学报, 2015,1(3): 278-283.
	YAO F Q , ZHANG Y , LIU Y X . Security and control for electromagnetic spectrum[J]. Journal of Command and Control, 2015,1(3): 278-283.
[2]	孙佳琛, 王金龙, 丁国如 ,等. 频谱知识图谱：面向未来频谱管理的智能引擎[J]. 通信学报, 2021,42(5): 1-12.
	SUN J C , WANG J L , DING G R ,et al. Spectrum knowledge graph:an intelligent engine facing future spectrum management[J]. Journal on Communications, 2021,42(5): 1-12.
[3]	孙仲康, 郭福成, 冯道旺 ,等. 单站无源定位跟踪技术[M]. 北京: 国防工业出版社, 2008.
	SUN Z K , GUO F C , FENG D W ,et al. Passive location and tracking technology by single observer[M]. Beijing: National Defense Industry Press, 2008.
[4]	刘聪锋 . 无源定位与跟踪[M]. 西安: 西安电子科技大学出版社, 2011.
	LIU C F . Passive location and tracking[M]. Xi'an: Xidian University Press, 2011.
[5]	BARTON D K . A half century of radar[J]. IEEE Transactions on Microwave Theory and Techniques, 1984,32(9): 1161-1170.
[6]	SKOLNIK M I . Fifty years of radar[J]. Proceedings of the IEEE, 1985,73(2): 182-197.
[7]	唐小明, 何友, 夏明革 . 基于机会发射的无源雷达系统发展评述[J]. 现代雷达, 2002,24(2): 1-6.
	TANG X M , HE Y , XIA M G . An overview of development of passive radar system based on transmitters of opportunity[J]. Modern Radar, 2002,24(2): 1-6.
[8]	陈新颖, 盛敏, 李博 ,等. 面向6G的无人机通信综述[J]. 电子与信息学报, 2022,44(3): 781-789.
	CHEN X Y , SHENG M , LI B ,et al. Survey on unmanned aerial vehicle communications for 6G[J]. Journal of Electronics ＆ Information Technology, 2022,44(3): 781-789.
[9]	ASGHAR S S A , SOLTANIZADEH H . Optimal trajectories for two UAVs in localization of multiple RF sources[J]. Transactions of the Institute of Measurement and Control, 2016,38(8): 908-916.
[10]	ASGHAR S S A , SOLTANIZADEH H . Single- and multi-UAV trajectory control in RF source localization[J]. Arabian Journal for Science and Engineering, 2017,42(2): 459-466.
[11]	WANG Z , CHEN G , BLASCH E ,et al. Jamming emitter localization with multiple UAVs equipped with smart antennas[J]. Proceedings of SPIE-The International Society for Optical Engineering, 2010,7696: 1-9.
[12]	WANG J , HINTON J , LIU J C L . RF based target search and localization with microUVA[C]// Proceedings of the 2016 International Conference on Computational Science and Computational Intelligence (CSCI). Piscataway:IEEE Press, 2016: 1077-1082.
[13]	TSUJI H , GRAY D , SUZUKI M ,et al. Radio location estimation experiment using array antennas for high altitude platforms[C]// Proceedings of the 18th IEEE International Symposium on Personal,Indoor and Mobile Radio Communications. Piscataway:IEEE Press, 2007: 1-5.
[14]	PACK D , YORK G , FIERRO R . Information-based cooperative control for multiple unmanned aerial vehicles[C]// Proceedings of the IEEE International Conference on Networking,Sensing and Control. Piscataway:IEEE Press, 2006: 446-450.
[15]	DOGANCAY K . UAV path planning for passive emitter localization[J]. IEEE Transactions on Aerospace and Electronic Systems, 2012,48(2): 1150-1166.
[16]	WANG L Y , HUANG Y . UAV-based estimation of direction of arrival:an approach based on image processing[C]// Proceedings of the 2020 International Conference on Wireless Communications and Signal Processing (WCSP). Piscataway:IEEE Press, 2020: 1165-1169.
[22]	WU S J . Illegal radio station localization with UAV-based Q-learning[J]. China Communications, 2018,15(12): 122-131.
[23]	JAAKKOLA T , SINGH S P , JORDAN M I . Reinforcement learning algorithm for partially observable Markov decision problems[C]// Proceedings of Advances in Neural Information Processing Systems. Massachusetts:MIT Press, 1995:7.
[24]	SHI W S , LI J L , WU H Q ,et al. Drone-cell trajectory planning and resource allocation for highly mobile networks:a hierarchical DRL approach[J]. IEEE Internet of Things Journal, 2021,8(12): 9800-9813.
[25]	BOUHAMED O , GHAZZAI H , BESBES H ,et al. Autonomous UAV navigation:a DDPG-based deep reinforcement learning approach[C]// Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS). Piscataway:IEEE Press, 2020: 1-5.
[17]	陈杰, 辛斌 . 有人/无人系统自主协同的关键科学问题[J]. 中国科学:信息科学, 2018,48(9): 1270-1274.
	CHEN J , XIN B . Key scientific problems in the autonomous cooperation of manned-unmanned systems[J]. Scientia Sinica (Informationis), 2018,48(9): 1270-1274.
[18]	LI Z , BRAUN T , ZHAO X H ,et al. A narrow-band indoor positioning system by fusing time and received signal strength via ensemble learning[J]. IEEE Access, 2018,6: 9936-9950.
[19]	VANSTEENWEGEN P , SOUFFRIAU W , OUDHEUSDEN D V . The orienteering problem:a survey[J]. European Journal of Operational Research, 2011,209(1): 1-10.
[20]	叶媛媛, 闵春平, 沈林成 . 多UCAV任务分配的混合遗传算法与约束处理[J]. 控制与决策, 2006,21(7): 781-786.
	YE Y Y , MIN C P , SHEN L C . Hybrid genetic algorithm and constraint handling for multiple UCAV mission assigning[J]. Control and Decision, 2006,21(7): 781-786.
[21]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518: 529-533.

有人机/无人机智能协同目标搜索和轨迹规划算法

Algorithm for intelligent collaborative target search and trajectory planning of MAV/UAV

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 21

相关文章 1

Metrics

推荐阅读 0