面向无人机集群协同感知的多智能体资源分配策略

doi:10.11959/j.issn.2096-3750.2023.00326

Abstract

Abstract:

Driven by the development of intelligent internet of things (IoT) technology, unmanned aerial vehicle (UAV) swarms have been widely used for sensing and monitoring in emergency and rescue scenarios.The UAVs automatically sense and discover mission targets in the mission area, recruiting neighboring UAVs to form perception and computation task groups to collaboratively complete the perception, acquisition and processing of data.However, repetitive sensory data and imbalance in the supply and demand of computational resources between multiple tasks cause additional computational and communication overheads and increase the end-to-end processing latency.To address this challenge, a multi-task resource allocation approach combining bionics and multi-agent independent reinforcement learning was proposed, making collaborative resource allocation decisions based on local task information.The method represents the resource requirements of individual tasks as situational information concentrations and dynamically updates the heterogeneous resource requirements of each task by spreading the situational information across task groups.At the same time, it combines multi-agent independent reinforcement learning methods for intelligent decision making in order to collaboratively allocate the heterogeneous resources of each task.Simulation results show that this solution can not only effectively reduce the task execution time, but also significantly improve the computational resource utilization.

Key words: UAV swarm, resource allocation, independent reinforcement learning, bionics, multi-agent

CLC Number:

TP393

Zhihong WANG, Supeng LENG, Kai XIONG. Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing[J]. Chinese Journal on Internet of Things, 2023, 7(1): 18-26.

Figures/Tables 6

实验参数	取值
无人机移动范围	$x_{i}, y_{i} \in [0, 3 000 m], i \in L$
任务区域单位长度	d=1 000m
无人机高度范围	$z_{i} \in [100 m, 500 m], i \in L$
总带宽	B=10 MHz
无人机CPU频率	$f_{i} \in [1 GHz, 2 GHz], i \in L$
点云规模	$N_{j} \in {3 \times 10^{4}, 5 \times 10^{4}, 8 \times 10^{4}}$
任务优先级	$o_{j} \in {1, 2, 3, 4, 5}$
任务最大执行时间	$t_{j}^{r} \in [300 s, 1 200 s]$
折扣因子γ	γ= 0.9
扩散跳数k	3
时隙?t	2s

References 30

[1]	ERDELJ M , NATALIZIO E , CHOWDHURY K R ,et al. Help from the sky:leveraging UAVs for disaster management[J]. IEEE Pervasive Computing, 2017,16(1): 24-32.
[2]	GUPTA L , JAIN R , VASZKUN G . Survey of important issues in UAV communication networks[J]. IEEE Communications Surveys ＆ Tutorials, 2016,18(2): 1123-1152.
[3]	MA H J , LIU Y L , REN Y H ,et al. Improved CNN classification method for groups of buildings damaged by earthquake,based on high resolution remote sensing images[J]. Remote Sensing, 2020,12(2): 260.
[4]	LI T Y , LENG S P , WAWG Z H ,et al. Intelligent resource allocation schemes for UAV-swarm-based cooperative sensing[J]. IEEE Internet of Things Journal, 2022,9(21): 21570-21582.
[5]	MEI H B , YANG K , LIU Q ,et al. Joint trajectory-resource optimization in UAV-enabled edge-cloud system with virtualized mobile clone[J]. IEEE Internet of Things Journal, 2020,7(7): 5906-5921.
[6]	SEID A M , BOATENG G O , ANOKYE S ,et al. Collaborative computation offloading and resource allocation in multi-UAV-assisted IoT networks:a deep reinforcement learning approach[J]. IEEE Internet of Things Journal, 2021,8(15): 12203-12218.
[7]	CHOI H L , BRUNET L , HOW J P . Consensus-based decentralized auctions for robust task allocation[J]. IEEE Transactions on Robotics, 2009,25(4): 912-926.
[8]	FU X W , FENG P , GAO X G . Swarm UAVs task and resource dynamic assignment algorithm based on task sequence mechanism[J]. IEEE Access, 2019(7): 41090-41100.
[9]	BAKOLAS E , LEE Y . Decentralized game theoretic control for dynamic task allocation problems for multi-agent systems[C]// Proceedings of 2021 American Control Conference (ACC). Piscataway:IEEE Press, 2021: 3228-3233.
[10]	ZHANG C Y , LI Q Y , ZHU Y Y ,et al. Dynamics of task allocation based on game theory in multi-agent systems[J]. IEEE Transactions on Circuits and Systems II:Express Briefs, 2019,66(6): 1068-1072.
[11]	BAKSHI S , FENG T H , YAN Z Y ,et al. A regularized quadratic programming approach to real-time scheduling of autonomous mobile robots in a prioritized task space[C]// Proceedings of 2019 American Control Conference (ACC). Piscataway:IEEE Press, 2019: 1361-1366.
[12]	ZHANG S H , ZHANG H L , DI B Y ,et al. Cellular cooperative unmanned aerial vehicle networks with sense-and-send protocol[J]. IEEE Internet of Things Journal, 2019,6(2): 1754-1767.
[13]	HOSSAIN A , CHAKRABARTI S , BISWAS P K . Impact of sensing model on wireless sensor network coverage[J]. IET Wireless Sensor Systems, 2012,2(3): 272.
[14]	CHAKRABORTY A , ROUT R R , CHAKRABARTI A ,et al. On network lifetime expectancy with realistic sensing and traffic generation model in wireless sensor networks[J]. IEEE Sensors Journal, 2013,13(7): 2771-2779.
[15]	SHAKHOV V V , KOO I . Experiment design for parameter estimation in probabilistic sensing models[J]. IEEE Sensors Journal, 2017,17(24): 8431-8437.
[16]	CHARLES R Q,HAOS , MOK C , et al . PointNet:deep learning on point sets for 3D classification and segmentation[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 77-85.
[17]	QI C R , YI L , SU H ,et al. PointNet++:deep hierarchical feature learning on point sets in a metric space[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York:ACM Press, 2017: 5105-5114.
[18]	ZHAO H T , WANG H J , WU W Y ,et al. Deployment algorithms for UAV airborne networks toward on-demand coverage[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2015-2031.
[19]	WANG B W , SUN Y J , DO-DUY T , ,et al. Adaptive D-hop connected dominating set in highly dynamic flying ad-hoc networks[J]. IEEE Transactions on Network Science and Engineering, 2021,8(3): 2651-2664.
[20]	VAW DYKE PARUNAK H , BRUECKNER S A , SAUTER J . Digital pheromones for coordination of unmanned vehicles[C]// Proceedings of the 1st International Conference on Environments for Multi-Agent Systems. New York:ACM Press, 2004: 246-263.
[21]	XU X , LI R P , ZHAO Z F ,et al. Stigmergic independent reinforcement learning for multiagent collaboration[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022,33(9): 4285-4299.
[22]	HOLLAND O E , . Multiagent systems:lessons from social insects and collective robotics[C]// Adaptation,Coevolution,and Learning in Multiagent Systems:Papers from the 1996 AAAI Spring Symposium.[S.l.:s.n.], 1996: 57-62.
[23]	DORIGO M , BONABEAU E , THERAULAZ G . Ant algorithms and stigmergy[J]. Future Generation Computer Systems, 2000,16(8): 851-871.
[24]	DI CARO G . AntNet:distributed stigmergetic control for communications networks[J]. Journal of Artificial Intelligence Research, 1998,9: 317-365.
[25]	ZHANG C , JIN S , XUE W ,et al. Independent reinforcement learning for weakly cooperative multiagent traffic control problem[J]. IEEE Transcations on Vehicular Technology, 2021,70(8): 7426-7436.
[26]	LANCTOT M , ZAMBALDI V , GRUSLYS A ,et al. A unified game-theoretic approach to multiagent reinforcement learning[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). New York:ACM Press, 2017: 4193-4206.
[27]	MATIGNON L , LAUREWT G J , LE FORT-PIAT N . Independent reinforcement learners in cooperative Markov games:a survey regarding coordination problems[J]. The Knowledge Engineering Review, 2012,27(1): 1-31.
[28]	ARULKUM ARAN K , DEISEWRC TH M P , BRUNDAGI M ,et al. Deep reinforcement learning:a brief survey[J]. IEEE Signal Processing Magazine, 2017,34(6): 26-38.
[29]	ZHANG T K , LEI J Y , LIU Y W ,et al. Trajectory optimization for UAV emergency communication with limited user equipment energy:a safe-DQN approach[J]. IEEE Transactions on Green Communications and Networking, 2021,5(3): 1236-1247.
[30]	FOERSTER J , NARDELLI N , FARQUHAR G ,et al. Stabilising experience replay for deep multi-agent reinforcement learning[C]// Proceedings of the 34th International Conference on Machine Learning. New York:ACM Press, 2017: 1146-1155.

Metrics

Recommended 0

No Suggested Reading articles found!

Multi-agent resource allocation strategy for UAV swarm-based cooperative sensing

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 30

Related Articles 15

Metrics

Recommended 0

[1]	Bei TANG, Qian WANG, Siguang CHEN. Radio frequency energy harvesting-combined collaborative energy-saving computation offloading mechanism [J]. Chinese Journal on Internet of Things, 2023, 7(1): 83-92.
[2]	Yao LIU, Yueyuan HE, Hongjing ZHOU, Chaoliang LI, Chuang LI. Partial computation offloading method based on joint resource allocation for mobile edge computing [J]. Chinese Journal on Internet of Things, 2023, 7(1): 140-148.
[3]	Biao ZHANG, Ximing WANG, Yifan XU, Wen LI, Hao HAN, Songyi LIU, Xueqiang CHEN. Multi-domain collaborative anti-jamming based on multi-agent deep reinforcement learning [J]. Chinese Journal on Internet of Things, 2022, 6(4): 104-116.
[4]	Jun SUN, Shangweikang ZHAO. Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things [J]. Chinese Journal on Internet of Things, 2022, 6(3): 82-90.
[5]	Yongjun XU, Haoke YANG, Yinghui YE, Qianbin CHEN, Guangyue LU. A survey on resource allocation in backscatter communication networks [J]. Chinese Journal on Internet of Things, 2021, 5(3): 56-69.
[6]	Qi ZHANG, Yuna JIANG, Xiaohu GE, Yonghui LI. Resource allocation based on optimal transport theory in IoT edge computing [J]. Chinese Journal on Internet of Things, 2021, 5(2): 60-70.
[7]	Mengrong LI, Huayu ZHU, Jin QI, Yanfei SUN. Optimal allocation model of logistics alliance resources based on willingness to participate [J]. Chinese Journal on Internet of Things, 2021, 5(1): 108-116.
[8]	Jiayu CAO,Supeng LENG,Ke ZHANG. Multi-agent driven collaborative decision mechanism of information fusion for autonomous driving vehicles [J]. Chinese Journal on Internet of Things, 2020, 4(3): 69-77.
[9]	Zhaorui WANG,Liang LIU,Hang LI,Shuguang CUI. Intelligent reflecting surface design for 6G-assited Internet of things [J]. Chinese Journal on Internet of Things, 2020, 4(2): 84-95.
[10]	Peng ZHOU,Jincheng XU,Bo YANG. Cross-domain task offloading and computing resource allocation for edge computation in industrial Internet of things [J]. Chinese Journal on Internet of Things, 2020, 4(2): 96-104.
[11]	Yueyue DAI,Ke ZHANG,Yan ZHANG. Blockchain empowered 6G [J]. Chinese Journal on Internet of Things, 2020, 4(1): 111-120.
[12]	Pengfei WANG,Boya DI,Lingyang SONG,Zhu HAN. Heterogeneous multi-layer mobile edge computing for 6G [J]. Chinese Journal on Internet of Things, 2020, 4(1): 121-130.
[13]	Zhipeng WANG,Bin CAO,Qinyu ZHANG. Research on computing resource allocation strategy for cloud radio access network based on demand forecasting [J]. Chinese Journal on Internet of Things, 2019, 3(4): 1-8.
[14]	Jiayu YANG,Jie HU,Supeng LENG,Kun YANG. Robust time resource allocation and multi-user selection algorithm for data and energy integrated communication network [J]. Chinese Journal on Internet of Things, 2019, 3(3): 34-40.
[15]	Kai XIONG,Supeng LENG,Ke ZHANG,Hao LIU. Research on heterogeneous radio access and resource allocation algorithm in vehicular fog computing [J]. Chinese Journal on Internet of Things, 2019, 3(2): 20-27.