有人机/无人机智能协同目标搜索和轨迹规划算法

doi:10.11959/j.issn.1000-436x.2024003

Abstract

Abstract:

Based on the manned aerial vehicle (MAV) / unmanned aerial vehicle (UAV) intelligent cooperation platform, the search of multiple interfered signal sources with unknown locations and trajectory planning were studied.Considering the real-time and dynamic nature of the search process, a MAV/UAV intelligent collaborative target search and trajectory planning (MUICTSTP) algorithm based on multi-agent deep reinforcement learning (MADRL) was proposed.Each UAV made online decision on trajectory planning by sensing the received interference signal strength (RISS) values, and then transmitted the sensing information and decision-making actions to the MAV to obtain the global evaluation.The simulation results show that the proposed algorithm exhibits better performance in long-term RISS, collision, and other aspects compared to other algorithms, and the learning strategy is better.

Key words: MAV/UAV, intelligent collaborative, MADRL, trajectory planning, RISS

CLC Number:

TN92

Zhuo LU, Qihui WU, Fuhui ZHOU. Algorithm for intelligent collaborative target search and trajectory planning of MAV/UAV[J]. Journal on Communications, 2024, 45(1): 31-40.

Figures/Tables 10

参数	含义
M ,U,N	有人机数量、无人机数量、干扰信号源数量
d_ij(t)	无人机i与干扰信号源 j在时刻t的距离
P_ij(t)	无人机i与干扰信号源 j在时刻t的瞬时功率
L,G ^T,G^R	路径损耗、发射天线增益、接收天线增益
$θ_{i}^{π}, θ_{m, i}^{Q}$	actor网络权重、critic网络权重
$θ_{i}^{π}^{'}, θ_{m, i}^{Q}^{'}$	target actor网络权重、target critic网络权重
B	经验回放池大小
N_b	样本大小
φ	样本索引
τ	软更新率
γ	折扣因子

References 21

[1]	姚富强, 张余, 柳永祥 . 电磁频谱安全与控制[J]. 指挥与控制学报, 2015,1(3): 278-283.
	YAO F Q , ZHANG Y , LIU Y X . Security and control for electromagnetic spectrum[J]. Journal of Command and Control, 2015,1(3): 278-283.
[2]	孙佳琛, 王金龙, 丁国如 ,等. 频谱知识图谱：面向未来频谱管理的智能引擎[J]. 通信学报, 2021,42(5): 1-12.
	SUN J C , WANG J L , DING G R ,et al. Spectrum knowledge graph:an intelligent engine facing future spectrum management[J]. Journal on Communications, 2021,42(5): 1-12.
[3]	孙仲康, 郭福成, 冯道旺 ,等. 单站无源定位跟踪技术[M]. 北京: 国防工业出版社, 2008.
	SUN Z K , GUO F C , FENG D W ,et al. Passive location and tracking technology by single observer[M]. Beijing: National Defense Industry Press, 2008.
[4]	刘聪锋 . 无源定位与跟踪[M]. 西安: 西安电子科技大学出版社, 2011.
	LIU C F . Passive location and tracking[M]. Xi'an: Xidian University Press, 2011.
[5]	BARTON D K . A half century of radar[J]. IEEE Transactions on Microwave Theory and Techniques, 1984,32(9): 1161-1170.
[6]	SKOLNIK M I . Fifty years of radar[J]. Proceedings of the IEEE, 1985,73(2): 182-197.
[7]	唐小明, 何友, 夏明革 . 基于机会发射的无源雷达系统发展评述[J]. 现代雷达, 2002,24(2): 1-6.
	TANG X M , HE Y , XIA M G . An overview of development of passive radar system based on transmitters of opportunity[J]. Modern Radar, 2002,24(2): 1-6.
[8]	陈新颖, 盛敏, 李博 ,等. 面向6G的无人机通信综述[J]. 电子与信息学报, 2022,44(3): 781-789.
	CHEN X Y , SHENG M , LI B ,et al. Survey on unmanned aerial vehicle communications for 6G[J]. Journal of Electronics ＆ Information Technology, 2022,44(3): 781-789.
[9]	ASGHAR S S A , SOLTANIZADEH H . Optimal trajectories for two UAVs in localization of multiple RF sources[J]. Transactions of the Institute of Measurement and Control, 2016,38(8): 908-916.
[10]	ASGHAR S S A , SOLTANIZADEH H . Single- and multi-UAV trajectory control in RF source localization[J]. Arabian Journal for Science and Engineering, 2017,42(2): 459-466.
[11]	WANG Z , CHEN G , BLASCH E ,et al. Jamming emitter localization with multiple UAVs equipped with smart antennas[J]. Proceedings of SPIE-The International Society for Optical Engineering, 2010,7696: 1-9.
[12]	WANG J , HINTON J , LIU J C L . RF based target search and localization with microUVA[C]// Proceedings of the 2016 International Conference on Computational Science and Computational Intelligence (CSCI). Piscataway:IEEE Press, 2016: 1077-1082.
[13]	TSUJI H , GRAY D , SUZUKI M ,et al. Radio location estimation experiment using array antennas for high altitude platforms[C]// Proceedings of the 18th IEEE International Symposium on Personal,Indoor and Mobile Radio Communications. Piscataway:IEEE Press, 2007: 1-5.
[14]	PACK D , YORK G , FIERRO R . Information-based cooperative control for multiple unmanned aerial vehicles[C]// Proceedings of the IEEE International Conference on Networking,Sensing and Control. Piscataway:IEEE Press, 2006: 446-450.
[15]	DOGANCAY K . UAV path planning for passive emitter localization[J]. IEEE Transactions on Aerospace and Electronic Systems, 2012,48(2): 1150-1166.
[16]	WANG L Y , HUANG Y . UAV-based estimation of direction of arrival:an approach based on image processing[C]// Proceedings of the 2020 International Conference on Wireless Communications and Signal Processing (WCSP). Piscataway:IEEE Press, 2020: 1165-1169.
[22]	WU S J . Illegal radio station localization with UAV-based Q-learning[J]. China Communications, 2018,15(12): 122-131.
[23]	JAAKKOLA T , SINGH S P , JORDAN M I . Reinforcement learning algorithm for partially observable Markov decision problems[C]// Proceedings of Advances in Neural Information Processing Systems. Massachusetts:MIT Press, 1995:7.
[24]	SHI W S , LI J L , WU H Q ,et al. Drone-cell trajectory planning and resource allocation for highly mobile networks:a hierarchical DRL approach[J]. IEEE Internet of Things Journal, 2021,8(12): 9800-9813.
[25]	BOUHAMED O , GHAZZAI H , BESBES H ,et al. Autonomous UAV navigation:a DDPG-based deep reinforcement learning approach[C]// Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS). Piscataway:IEEE Press, 2020: 1-5.
[17]	陈杰, 辛斌 . 有人/无人系统自主协同的关键科学问题[J]. 中国科学:信息科学, 2018,48(9): 1270-1274.
	CHEN J , XIN B . Key scientific problems in the autonomous cooperation of manned-unmanned systems[J]. Scientia Sinica (Informationis), 2018,48(9): 1270-1274.
[18]	LI Z , BRAUN T , ZHAO X H ,et al. A narrow-band indoor positioning system by fusing time and received signal strength via ensemble learning[J]. IEEE Access, 2018,6: 9936-9950.
[19]	VANSTEENWEGEN P , SOUFFRIAU W , OUDHEUSDEN D V . The orienteering problem:a survey[J]. European Journal of Operational Research, 2011,209(1): 1-10.
[20]	叶媛媛, 闵春平, 沈林成 . 多UCAV任务分配的混合遗传算法与约束处理[J]. 控制与决策, 2006,21(7): 781-786.
	YE Y Y , MIN C P , SHEN L C . Hybrid genetic algorithm and constraint handling for multiple UCAV mission assigning[J]. Control and Decision, 2006,21(7): 781-786.
[21]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518: 529-533.

Metrics

Recommended 0

No Suggested Reading articles found!

Algorithm for intelligent collaborative target search and trajectory planning of MAV/UAV

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 21

Related Articles 15

Metrics

Recommended 0

[1]	Lingjin KONG, Kai MEI, Xiaoran LIU, Jun XIONG, Haitao ZHAO, Jibo WEI. Design and implementation of online learning assisted intelligent receiver [J]. Journal on Communications, 2024, 45(1): 18-30.
[2]	Hao YIN, Jibo WEI, Haitao ZHAO, Jiao ZHANG, Haijun WANG, Baoquan REN. Intelligent communication and networking key technologies for manned/unmanned cooperation: states-of-the-art and trends [J]. Journal on Communications, 2024, 45(1): 1-17.
[3]	Yuntao WANG, Zhou SU, Qichao XU, Yiliang LIU, Haixia PENG, Hao LUAN. Secure and collaborative spectrum sensing scheme based on audit game [J]. Journal on Communications, 2023, 44(12): 1-14.
[4]	Yutong ZHANG, Yuming PENG, Boya DI, Lingyang SONG. Heterogeneous resource cooperative game in space-ground computing power network [J]. Journal on Communications, 2023, 44(12): 15-27.
[5]	Yangyang NIU, Zhiqing WEI, Zhiyong FENG. Space-time spectrum sharing based on game theory:dynamic access and penalty strategy [J]. Journal on Communications, 2023, 44(12): 28-38.
[6]	Zining WANG, Min LIN, Fei LI, Lve HAN, Weiping ZHU. Distributed robust beamforming algorithm for spectral coexistence in satellite and high altitude platform networks [J]. Journal on Communications, 2023, 44(12): 39-49.
[7]	Bai ZHAO, Min LIN, Shengjie XIAO, Ming CHENG, Yifan FENG, Lvxi YANG. Rate splitting based robust secure transmission scheme in RIS-assisted satellite-terrestrial integrated network [J]. Journal on Communications, 2023, 44(12): 50-60.
[8]	Ruyan WANG, Xianyi YE, Peng HE, Yaping CUI, Dapeng WU, Fedotov Alexander. LEO satellite constellation design with multi-QoS guarantee for non-terrestrial network [J]. Journal on Communications, 2023, 44(12): 61-77.
[9]	Fanglin GU, Jinlin PENG, Yuzhen HUANG, Shengqun CAO, Haitao ZHAO, Jibo WEI. Novel direct sequence spread spectrum communication method [J]. Journal on Communications, 2023, 44(12): 78-85.
[10]	Cheng ZHANG, Jiaye ZHU, Zening LIU, Yongming HUANG. Multi-agent reinforcement learning based dynamic optimization algorithm of CRE offset for heterogeneous networks [J]. Journal on Communications, 2023, 44(12): 86-98.
[11]	Junbing LI, Youjun ZENG, Xiaoping ZENG, Guojun LI, Chenxi BAI. Short-term prediction network for short-wave MUF based on model-data dual-driven [J]. Journal on Communications, 2023, 44(12): 99-111.
[12]	Bowen ZHAO, Yao ZHU, Yang XIAO, Qingqi PEI, Xiaoguo LI, Ximeng LIU. Rational-security and fair two-party comparison protocol [J]. Journal on Communications, 2023, 44(12): 112-123.
[13]	Yun ZHANG, Jing ZHOU, Jingwei HUANG, Shujuan YU, Liya HUANG. Channel estimation for OFDM system based on deep learning [J]. Journal on Communications, 2023, 44(12): 124-133.
[14]	Zhiguo SUN, Zhao WANG, Zengmao CHEN, Rongchen SUN. Double reputation and multi-angle weight-based cooperative spectrum sensing method against SSDF attacks [J]. Journal on Communications, 2023, 44(12): 134-145.
[15]	Jing ZHANG, Lin MA, Chulong LIANG, Hongxu GAO. Massive unsourced multiple access scheme based on block sequence codebook and compressed sensing [J]. Journal on Communications, 2023, 44(12): 171-180.