无人机集群联合拓扑控制的智能路由规划方法

doi:10.11959/j.issn.1000-436x.2024032

Abstract

Abstract:

Existing routing protocols without awareness of the topology causes excessive retransmissions, energy holes, and long delay, data routing performance was seriously deteriorated.Considering the relation of topology and routing, an intelligent route planning with jointing topology control (IRPJTC) method was proposed.IRPJTC consisted of two part, the virtual force-based adaptive topology control (VFATC), and the PPO-based geographic routing protocol (PPO-GRP).Based on neighbor’s mobility information, the distance between UAVs was adaptively adjusted by VFATC to provide stable links between UAVs.Combined with link stability metric in VFATC, end-to-end delay and energy consumption, a multi-objective reward function was designed by PPO-GRP to train optimal routing strategy.According to the performance study, the proposed IRPJTC reduces existing routing protocols by 12.11% of end-to-end delay, and 4.56% of energy consumption, and has a better energy balance ability.

Key words: UAV swarm, routing protocol, topology control, proximal policy optimization, deep reinforcement learning

CLC Number:

TN915.04

Zhi YAN, Zhenglun YI, Bo OUYANG, Yaonan WANG. Intelligent route planning method with jointing topology control of UAV swarm[J]. Journal on Communications, 2024, 45(2): 137-149.

Figures/Tables 14

参数	含义
$T = {t_{1}, t_{2}, \dots, t_{m - 1}, t_{m}}$	区域监视总任务时间与时隙划分
τ	每个时隙的长度
$U = {u_{1}, u_{2}, \dots, u_{k}}$	无人机集合
${\vec{p}}_{i} (t_{n})$	无人机u_i在时隙t_n 的位置向量
${\vec{p}}_{BS}$	地面站的位置坐标
${\vec{v}}_{i} (t_{n})$	无人机u_i在时隙t_n 的速度向量
${\vec{a}}_{i} (t_{n})$	无人机u_i在时隙t_n 的加速度向量
R_r	安全距离约束范围（斥力范围）
R_a	无人机最大传输距离（引力范围）
R_BS	无人机与地面站的通信范围
N_i	u _i的单跳邻居集
$N_{i}^{a}$	u _i引力范围内的单跳邻居集
$N_{i}^{r}$	u _i斥力范围内的单跳邻居集
$d_{i j} (t_{n})$	u _i与u_j在时隙t_n 的欧氏距离
$f_{i j}^{Time}$	单跳数据包传输时间
$f_{i j}^{Speed}$	单跳数据包传输速度
LD _ij	链路寿命

参数	含义	值
c₁	评估网络目标函数所占权重值	-0.5
c₂	策略模型的熵所占权重值	-0.01
$℧_{\max}$	关于动作值概率分布的标准差的最大值	0.6
$℧_{\min}$	关于动作值概率分布的标准差的最小值	0.1
$α_{℧}$	关于动作值概率分布标准差的衰减因子	0.999 5
Episodes	仿真回合数	500
Baffer-Size	缓冲区D的大小	2 000
n_update	网络连续更新次数	16
ε	PPO的裁剪参数	0.1
γ	计算奖励期望的折扣系数	0.95
策略网络学习率		0.000 03
评估网络学习率		0.000 1

References 19

[1]	CHENG X , DONG C , DAI H P ,et al. MOOC:a mobility control based clustering scheme for area coverage in FANETs[C]// Proceedings of the 2018 IEEE 19th International Symposium on “A World of Wireless,Mobile and Multimedia Networks”. Piscataway:IEEE Press, 2018: 14-22.
[2]	LIN Z J , LIU H H T , WOTTON M . Kalman filter-based large-scale wildfire monitoring with a system of UAVs[J]. IEEE Transactions on Industrial Electronics, 2019,66(1): 606-615.
[3]	WANG H J , ZHAO H T , WU W Y ,et al. Deployment algorithms of flying base stations:5G and beyond with UAVs[J]. IEEE Internet of Things Journal, 2019,6(6): 10009-10027.
[4]	许文俊, 吴思雷, 王凤玉 ,等. 基于多智能体强化学习的大规模灾后用户分布式覆盖优化[J]. 通信学报, 2022,43(8): 1-16.
	XU W J , WU S L , WANG F Y ,et al. Large-scale post-disaster user distributed coverage optimization based on multi-agent reinforcement learning[J]. Journal on Communications, 2022,43(8): 1-16.
[5]	余雪勇, 邱礼翔, 宋家宁 ,等. 无人机辅助边缘计算中安全通信与能效优化策略[J]. 通信学报, 2023,44(3): 45-54.
	YU X Y , QIU L X , SONG J N ,et al. Security communication and energy efficiency optimization strategy in UAV-aided edge computing[J]. Journal on Communications, 2023,44(3): 45-54.
[6]	ALAM M M , ARAFAT M Y , MOH S ,et al. Topology control algorithms in multi-unmanned aerial vehicle networks:an extensive survey[J]. Journal of Network and Computer Applications, 2022,207:103495.
[7]	KIM D Y , LEE J W . Joint mission assignment and topology management in the mission-critical FANET[J]. IEEE Internet of Things Journal, 2020,7(3): 2368-2385.
[8]	TROTTA A , MONTECCHIARI L , FELICE M D ,et al. A GPS-free flocking model for aerial mesh deployments in disaster-recovery scenarios[J]. IEEE Access, 2020,8: 91558-91573.
[9]	ZHAO H T , WEI J B , HUANG S C ,et al. Regular topology formation based on artificial forces for distributed mobile robotic networks[J]. IEEE Transactions on Mobile Computing, 2019,18(10): 2415-2429.
[10]	KARP B , KUNG H T . GPSR:greedy perimeter stateless routing for wireless networks[C]// Proceedings of the 6th Annual International Conference on Mobile Computing and Networking. New York:ACM Press, 2000: 243-254.
[11]	JUNG W S , YIM J , KO Y B . QGeo:Q-learning-based geographic ad hoc routing protocol for unmanned robotic networks[J]. IEEE Communications Letters, 2017,21(10): 2258-2261.
[12]	LIU J M , WANG Q , HE C T ,et al. QMR:Q-learning based multi-objective optimization routing protocol for flying ad hoc networks[J]. Computer Communications, 2020,150: 304-316.
[13]	张雅楠, 仇洪冰 . 基于深度强化学习的无人机可信地理位置路由协议[J]. 电子与信息学报, 2022,44(12): 4211-4217.
	ZHANG Y N , QIU H B . Trusted geographic routing protocol based on deep reinforcement learning for unmanned aerial vehicle network[J]. Journal of Electronics ＆ Information Technology, 2022,44(12): 4211-4217.
[14]	LIN D P , PENG T , ZUO P L ,et al. Deep-reinforcement-learning-based intelligent routing strategy for FANETs[J]. Symmetry, 2022,14(9): 1787.
[15]	BAI Y J , ZHANG X , YU D J ,et al. A deep reinforcement learning-based geographic packet routing optimization[J]. IEEE Access, 2022,10: 108785-108796.
[16]	陈辉 . 无线 AdHoc 路由算法和拓扑控制算法研究[D]. 西安:长安大学, 2014.
	CHEN H . Research on wireless AdHoc routing algorithm and topology control algorithm[D]. Xi’an:Changan University, 2014.
[17]	刘航 . 资源受限卫星网络拓扑与路由规划方法研究[D]. 西安:西安电子科技大学, 2022.
	LIU H . Research on topology and routing planning of resource-constrained satellite networks[D]. Xi’an:Xidian University, 2022.
[18]	WU Y L , ZHANG B , YANG S S ,et al. Energy-efficient joint communication-motion planning for relay-assisted wireless robot surveillance[C]// Proceedings of the IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2017: 1-9.
[19]	LIU Q W , ZHOU S L , GIANNAKIS G B . Cross-layer combining of queuing with adaptive modulation and coding over wireless links[C]// Proceedings of the IEEE Military Communications Conference. Piscataway:IEEE Press, 2004: 717-722.

Metrics

Recommended 0

No Suggested Reading articles found!

参数	含义	值
M	无人机数量	30～80
τ /s	每个时隙的长度	0.1
σ /s	Hello包广播间隔	0.5
ψ	路径损耗指数	3
SINR_th	SINR阈值	1
R_a/m	最大距离（吸引距离）	150
R_r/m	最小距离（排斥距离）	100
B/MHz	带宽	20
E_total/J	无人机初始能量	20 000

Intelligent route planning method with jointing topology control of UAV swarm

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 19

Related Articles 15

Metrics

Recommended 0

[1]	Bin LI, Sicong PENG, Zesong FEI. Beamforming and resource optimization in UAV integrated sensing and communication network with edge computing [J]. Journal on Communications, 2023, 44(9): 228-237.
[2]	Qiaoshou LIU, Xiong ZHOU, Shuang LIU, Yifeng DENG. Adaptive pilot design for OFDM based on deep reinforcement learning [J]. Journal on Communications, 2023, 44(9): 104-114.
[3]	Runzi LIU, Tianci MA, Weihua WU, Chenhong YAO, Qinghai YANG. Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning [J]. Journal on Communications, 2023, 44(7): 207-217.
[4]	Ningjiang CHEN, Linming LIAN, Pingjie OU, Xuemei YUAN. D2D cooperative caching strategy based on graph collaborative filtering model [J]. Journal on Communications, 2023, 44(7): 136-148.
[5]	Feng ZENG, Zheng ZHANG, Zhigang CHEN. Computation offloading and resource allocation strategy based on deep reinforcement learning [J]. Journal on Communications, 2023, 44(7): 124-135.
[6]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[7]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[8]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[9]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[10]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[11]	Yanfei SUN, Jiazheng YIN, Jin QI, Xiaoxuan HU, Mengting CHEN, Zhenjiang DONG. Topology control based on dynamic graph embedding in Internet of vehicles [J]. Journal on Communications, 2022, 43(6): 133-142.
[12]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[13]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[14]	Yu CHEN, Lianxing JIA. Two-layer grouped Byzantine fault tolerance algorithm for UAV swarm [J]. Journal on Communications, 2022, 43(1): 96-103.
[15]	Li’na DU, Li ZHUO, Shuo YANG, Jiafeng LI, Jing ZHANG. Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services [J]. Journal on Communications, 2021, 42(9): 205-217.