面向无人机集群协同感知的多智能体资源分配策略

doi:10.11959/j.issn.2096-3750.2023.00326

物联网学报 ›› 2023, Vol. 7 ›› Issue (1): 18-26.doi: 10.11959/j.issn.2096-3750.2023.00326

• 专题：面向情景的智能网络与按需组网 • 上一篇下一篇

面向无人机集群协同感知的多智能体资源分配策略

王志宏¹^,², 冷甦鹏¹^,², 熊凯¹^,²

¹ 电子科技大学信息与通信工程学院，四川成都 611731
² 电子科技大学（深圳）高等研究院，广东深圳 518110

修回日期:2022-12-29 出版日期:2023-03-30 发布日期:2023-03-01
作者简介:王志宏（1997- ），男，电子科技大学博士生，主要研究方向为无人机集群资源分配、语义通信和机器学习
冷甦鹏（1973- ），男，电子科技大学教授、博士生导师，主要研究方向为物联网、车联网、新一代宽带无线网络、无线自组织网络、智能交通信息系统的资源管理、介质访问控制、路由、组网与互联、智能算法理论及技术应用等
熊凯（1991- ），男，电子科技大学在站博士后，主要研究方向为车联网资源分配、移动边缘计算和机器学习
基金资助:
国家自然科学基金资助项目(62171104)

Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing

Zhihong WANG¹^,², Supeng LENG¹^,², Kai XIONG¹^,²

¹ School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
² Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen 518110, China

Revised:2022-12-29 Online:2023-03-30 Published:2023-03-01
Supported by:
The National Natural Science Foundation of China(62171104)

摘要/Abstract

摘要：

在智能物联网技术发展的推动下，无人机集群已广泛用于应急、救援等场景的感知监测。无人机在任务区域自动感知发现任务目标，邻近无人机组成协作感知与协作计算任务群组，协同完成数据的感知、采集和处理。然而，重复的感知资源以及多任务间的计算资源供需不平衡，会造成额外的计算与通信开销，增大端到端处理时延。为了应对这一挑战，提出了一种结合仿生学和多智能体独立强化学习的多任务资源分配策略，基于局部的任务信息进行资源协同分配决策。该方法用任务情景信息浓度表示各个任务的资源需求，并通过情景信息在各任务群组间的扩散，动态更新各任务异构资源需求。同时，结合独立多智能体强化学习方法进行智能决策，以对各任务异构资源进行智能协同分配。仿真结果表明，所提方案不仅能够有效缩短任务执行时间，还可显著提高计算资源利用率。

关键词: 无人机集群, 资源分配, 独立强化学习, 仿生学, 多智能体

Abstract:

Driven by the development of intelligent internet of things (IoT) technology, unmanned aerial vehicle (UAV) swarms have been widely used for sensing and monitoring in emergency and rescue scenarios.The UAVs automatically sense and discover mission targets in the mission area, recruiting neighboring UAVs to form perception and computation task groups to collaboratively complete the perception, acquisition and processing of data.However, repetitive sensory data and imbalance in the supply and demand of computational resources between multiple tasks cause additional computational and communication overheads and increase the end-to-end processing latency.To address this challenge, a multi-task resource allocation approach combining bionics and multi-agent independent reinforcement learning was proposed, making collaborative resource allocation decisions based on local task information.The method represents the resource requirements of individual tasks as situational information concentrations and dynamically updates the heterogeneous resource requirements of each task by spreading the situational information across task groups.At the same time, it combines multi-agent independent reinforcement learning methods for intelligent decision making in order to collaboratively allocate the heterogeneous resources of each task.Simulation results show that this solution can not only effectively reduce the task execution time, but also significantly improve the computational resource utilization.

Key words: UAV swarm, resource allocation, independent reinforcement learning, bionics, multi-agent

中图分类号:

TP393

王志宏, 冷甦鹏, 熊凯. 面向无人机集群协同感知的多智能体资源分配策略[J]. 物联网学报, 2023, 7(1): 18-26.

Zhihong WANG, Supeng LENG, Kai XIONG. Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing[J]. Chinese Journal on Internet of Things, 2023, 7(1): 18-26.

图/表 6

图1

表1

实验参数"

实验参数	取值
无人机移动范围	$x_{i}, y_{i} \in [0, 3 000 m], i \in L$
任务区域单位长度	d=1 000m
无人机高度范围	$z_{i} \in [100 m, 500 m], i \in L$
总带宽	B=10 MHz
无人机CPU频率	$f_{i} \in [1 GHz, 2 GHz], i \in L$
点云规模	$N_{j} \in {3 \times 10^{4}, 5 \times 10^{4}, 8 \times 10^{4}}$
任务优先级	$o_{j} \in {1, 2, 3, 4, 5}$
任务最大执行时间	$t_{j}^{r} \in [300 s, 1 200 s]$
折扣因子γ	γ= 0.9
扩散跳数k	3
时隙?t	2s

表1

图2

图3

图4

图5

参考文献 30

[1]	ERDELJ M , NATALIZIO E , CHOWDHURY K R ,et al. Help from the sky:leveraging UAVs for disaster management[J]. IEEE Pervasive Computing, 2017,16(1): 24-32.
[2]	GUPTA L , JAIN R , VASZKUN G . Survey of important issues in UAV communication networks[J]. IEEE Communications Surveys ＆ Tutorials, 2016,18(2): 1123-1152.
[3]	MA H J , LIU Y L , REN Y H ,et al. Improved CNN classification method for groups of buildings damaged by earthquake,based on high resolution remote sensing images[J]. Remote Sensing, 2020,12(2): 260.
[4]	LI T Y , LENG S P , WAWG Z H ,et al. Intelligent resource allocation schemes for UAV-swarm-based cooperative sensing[J]. IEEE Internet of Things Journal, 2022,9(21): 21570-21582.
[5]	MEI H B , YANG K , LIU Q ,et al. Joint trajectory-resource optimization in UAV-enabled edge-cloud system with virtualized mobile clone[J]. IEEE Internet of Things Journal, 2020,7(7): 5906-5921.
[6]	SEID A M , BOATENG G O , ANOKYE S ,et al. Collaborative computation offloading and resource allocation in multi-UAV-assisted IoT networks:a deep reinforcement learning approach[J]. IEEE Internet of Things Journal, 2021,8(15): 12203-12218.
[7]	CHOI H L , BRUNET L , HOW J P . Consensus-based decentralized auctions for robust task allocation[J]. IEEE Transactions on Robotics, 2009,25(4): 912-926.
[8]	FU X W , FENG P , GAO X G . Swarm UAVs task and resource dynamic assignment algorithm based on task sequence mechanism[J]. IEEE Access, 2019(7): 41090-41100.
[9]	BAKOLAS E , LEE Y . Decentralized game theoretic control for dynamic task allocation problems for multi-agent systems[C]// Proceedings of 2021 American Control Conference (ACC). Piscataway:IEEE Press, 2021: 3228-3233.
[10]	ZHANG C Y , LI Q Y , ZHU Y Y ,et al. Dynamics of task allocation based on game theory in multi-agent systems[J]. IEEE Transactions on Circuits and Systems II:Express Briefs, 2019,66(6): 1068-1072.
[11]	BAKSHI S , FENG T H , YAN Z Y ,et al. A regularized quadratic programming approach to real-time scheduling of autonomous mobile robots in a prioritized task space[C]// Proceedings of 2019 American Control Conference (ACC). Piscataway:IEEE Press, 2019: 1361-1366.
[12]	ZHANG S H , ZHANG H L , DI B Y ,et al. Cellular cooperative unmanned aerial vehicle networks with sense-and-send protocol[J]. IEEE Internet of Things Journal, 2019,6(2): 1754-1767.
[13]	HOSSAIN A , CHAKRABARTI S , BISWAS P K . Impact of sensing model on wireless sensor network coverage[J]. IET Wireless Sensor Systems, 2012,2(3): 272.
[14]	CHAKRABORTY A , ROUT R R , CHAKRABARTI A ,et al. On network lifetime expectancy with realistic sensing and traffic generation model in wireless sensor networks[J]. IEEE Sensors Journal, 2013,13(7): 2771-2779.
[15]	SHAKHOV V V , KOO I . Experiment design for parameter estimation in probabilistic sensing models[J]. IEEE Sensors Journal, 2017,17(24): 8431-8437.
[16]	CHARLES R Q,HAOS , MOK C , et al . PointNet:deep learning on point sets for 3D classification and segmentation[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 77-85.
[17]	QI C R , YI L , SU H ,et al. PointNet++:deep hierarchical feature learning on point sets in a metric space[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York:ACM Press, 2017: 5105-5114.
[18]	ZHAO H T , WANG H J , WU W Y ,et al. Deployment algorithms for UAV airborne networks toward on-demand coverage[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2015-2031.
[19]	WANG B W , SUN Y J , DO-DUY T , ,et al. Adaptive D-hop connected dominating set in highly dynamic flying ad-hoc networks[J]. IEEE Transactions on Network Science and Engineering, 2021,8(3): 2651-2664.
[20]	VAW DYKE PARUNAK H , BRUECKNER S A , SAUTER J . Digital pheromones for coordination of unmanned vehicles[C]// Proceedings of the 1st International Conference on Environments for Multi-Agent Systems. New York:ACM Press, 2004: 246-263.
[21]	XU X , LI R P , ZHAO Z F ,et al. Stigmergic independent reinforcement learning for multiagent collaboration[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022,33(9): 4285-4299.
[22]	HOLLAND O E , . Multiagent systems:lessons from social insects and collective robotics[C]// Adaptation,Coevolution,and Learning in Multiagent Systems:Papers from the 1996 AAAI Spring Symposium.[S.l.:s.n.], 1996: 57-62.
[23]	DORIGO M , BONABEAU E , THERAULAZ G . Ant algorithms and stigmergy[J]. Future Generation Computer Systems, 2000,16(8): 851-871.
[24]	DI CARO G . AntNet:distributed stigmergetic control for communications networks[J]. Journal of Artificial Intelligence Research, 1998,9: 317-365.
[25]	ZHANG C , JIN S , XUE W ,et al. Independent reinforcement learning for weakly cooperative multiagent traffic control problem[J]. IEEE Transcations on Vehicular Technology, 2021,70(8): 7426-7436.
[26]	LANCTOT M , ZAMBALDI V , GRUSLYS A ,et al. A unified game-theoretic approach to multiagent reinforcement learning[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). New York:ACM Press, 2017: 4193-4206.
[27]	MATIGNON L , LAUREWT G J , LE FORT-PIAT N . Independent reinforcement learners in cooperative Markov games:a survey regarding coordination problems[J]. The Knowledge Engineering Review, 2012,27(1): 1-31.
[28]	ARULKUM ARAN K , DEISEWRC TH M P , BRUNDAGI M ,et al. Deep reinforcement learning:a brief survey[J]. IEEE Signal Processing Magazine, 2017,34(6): 26-38.
[29]	ZHANG T K , LEI J Y , LIU Y W ,et al. Trajectory optimization for UAV emergency communication with limited user equipment energy:a safe-DQN approach[J]. IEEE Transactions on Green Communications and Networking, 2021,5(3): 1236-1247.
[30]	FOERSTER J , NARDELLI N , FARQUHAR G ,et al. Stabilising experience replay for deep multi-agent reinforcement learning[C]// Proceedings of the 34th International Conference on Machine Learning. New York:ACM Press, 2017: 1146-1155.

面向无人机集群协同感知的多智能体资源分配策略

Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 30

相关文章 15

Metrics

推荐阅读 0

[1]	汤蓓, 王倩, 陈思光. 融合射频能量采集的协同节能计算迁移研究[J]. 物联网学报, 2023, 7(1): 83-92.
[2]	刘耀, 何岳园, 周红静, 李超良, 李闯. 移动边缘计算中基于资源联合分配的部分计算卸载方法[J]. 物联网学报, 2023, 7(1): 140-148.
[3]	张彪, 汪西明, 徐逸凡, 李文, 韩昊, 刘松仪, 陈学强. 基于多智能体深度强化学习的多域协同抗干扰方法研究[J]. 物联网学报, 2022, 6(4): 104-116.
[4]	孙君, 赵尚维康. 工业物联网中基于Sarsa算法的节能计算卸载方案[J]. 物联网学报, 2022, 6(3): 82-90.
[5]	陈九九, 郭彩丽, 冯春燕, 刘传宏. 智能网联环境下面向语义通信的资源分配[J]. 物联网学报, 2022, 6(3): 47-57.
[6]	徐勇军, 杨浩克, 叶迎晖, 陈前斌, 卢光跃. 反向散射通信网络资源分配综述[J]. 物联网学报, 2021, 5(3): 56-69.
[7]	张琪, 蒋宇娜, 葛晓虎, 李永会. 基于最优运输理论的物联网边缘计算资源优化机制[J]. 物联网学报, 2021, 5(2): 60-70.
[8]	曹佳钰,冷甦鹏,张科. 面向自动驾驶应用的车联多智能体信息融合协同决策机制研究[J]. 物联网学报, 2020, 4(3): 69-77.
[9]	周鹏,徐金城,杨博. 工业物联网中基于边缘计算的跨域计算资源分配与任务卸载[J]. 物联网学报, 2020, 4(2): 96-104.
[10]	代玥玥,张科,张彦. 区块链赋能6G[J]. 物联网学报, 2020, 4(1): 111-120.
[11]	王鹏飞,邸博雅,宋令阳,韩竹. 6G异构边缘计算[J]. 物联网学报, 2020, 4(1): 121-130.
[12]	王志朋,曹斌,张钦宇. 基于需求预测的云无线接入网计算资源分配策略研究[J]. 物联网学报, 2019, 3(4): 1-8.
[13]	杨佳雨,胡杰,冷甦鹏,杨鲲. 一种具有顽健性的无线数能网络的时隙资源分配和多用户选择算法[J]. 物联网学报, 2019, 3(3): 34-40.
[14]	熊凯,冷甦鹏,张可,刘浩. 车联雾计算中的异构接入与资源分配算法研究[J]. 物联网学报, 2019, 3(2): 20-27.
[15]	刘鑫. 分簇认知物联网联合资源分配算法[J]. 物联网学报, 2019, 3(1): 14-19.