面向多波束卫星系统的波束跳变与覆盖控制联合优化算法

doi:10.11959/j.issn.1000-436x.2023076

通信学报 ›› 2023, Vol. 44 ›› Issue (4): 78-86.doi: 10.11959/j.issn.1000-436x.2023076

面向多波束卫星系统的波束跳变与覆盖控制联合优化算法

许国良¹^,², 谭峰¹, 冉泳屹¹^,², 陈丰¹

¹ 重庆邮电大学通信与信息工程学院，重庆 400065
² 重庆邮电大学电子信息与网络工程研究院，重庆 400065

修回日期:2023-02-01 出版日期:2023-04-25 发布日期:2023-04-01
作者简介:许国良（1973- ），男，浙江金华人，博士，重庆邮电大学教授，主要研究方向为大数据信息挖掘、智能分析、传感检测、感知系统和传感网络等领域的新技术和应用
谭峰（1998- ），男，重庆人，重庆邮电大学硕士生，主要研究方向为卫星通信、资源分配、路由算法等
冉泳屹（1986- ），男，重庆人，重庆邮电大学讲师、硕士生导师，主要研究方向为深度强化学习、深度学习、随机优化等方法在计算机系统和通信网络的应用
陈丰（1997- ），男，安徽池州人，重庆邮电大学硕士生，主要研究方向为卫星通信、路由算法等
基金资助:
国家自然科学基金资助项目(62171072);国家自然科学基金资助项目(62172064);国家自然科学基金资助项目(62003067);重庆市自然科学基金资助项目(cstc2021jcyj-msxmX0586)

Joint beam hopping and coverage control optimization algorithm for multibeam satellite system

Guoliang XU¹^,², Feng TAN¹, Yongyi RAN¹^,², Feng CHEN¹

¹ School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
² Institute of Electronic Information and Network Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Revised:2023-02-01 Online:2023-04-25 Published:2023-04-01
Supported by:
The National Natural Science Foundation of China(62171072);The National Natural Science Foundation of China(62172064);The National Natural Science Foundation of China(62003067);The Natural Science Foundation of Chongqing(cstc2021jcyj-msxmX0586)

摘要/Abstract

摘要：

为了提升多波束卫星（MBS）系统的性能，提出了一种基于深度强化学习联合优化MBS的波束跳变和覆盖控制（BHCC）算法。首先，将多波束卫星中的资源分配问题转换为多目标优化问题，以最大化多波束卫星系统的系统吞吐量，最小化丢包率；其次，将多波束卫星环境表示为多维矩阵，并将目标问题建模为考虑随机通信需求的马尔可夫决策过程；最后，结合深度强化学习强大的特征提取能力和学习能力对目标问题进行求解。此外，提出了一种单智能体轮询复用机制以减少搜索空间，降低收敛难度，加速BHCC算法的训练。仿真结果表明，相对于遗传算法、贪婪算法及随机算法，BHCC算法不仅能提高MBS的吞吐量，而且能降低系统的丢包率；相对于不考虑自适应波束覆盖范围的深度强化学习算法，BHCC算法在不同通信场景下的性能更优异。

关键词: 多波束卫星, 深度强化学习, 波束跳变技术, 波束覆盖控制

Abstract:

To improve the performance of multibeam satellite (MBS) systems, a deep reinforcement learning-based algorithm to jointly optimize the beam hopping and coverage control (BHCC) algorithm for MBS was proposed.Firstly, the resource allocation problem in MBS was transformed to a multi-objective optimization problem with the objective maximizing the system throughput and minimizing the packet loss rate of the MBS.Secondly, the MBS environment was characterized as a multi-dimensional matrix, and the objective problem was modelled as a Markov decision process considering stochastic communication requirements.Finally, the objective problem was solved by combining the powerful feature extraction and learning capabilities of deep reinforcement learning.In addition, a single-intelligence polling multiplexing mechanism was proposed to reduce the search space and convergence difficulty and accelerate the training of BHCC.Compared with the genetic algorithm, the simulation results show that BHCC improves the throughput of MBS and reduces the packet loss rate of the system, greedy algorithm, and random algorithm.Besides, BHCC performs better in different communication scenarios compared with a deep reinforcement learning algorithm, which do not consider the adaptive beam coverage.

Key words: multibeam satellite, deep reinforcement learning, beam hopping technology, beam coverage control

中图分类号:

TN92

许国良, 谭峰, 冉泳屹, 陈丰. 面向多波束卫星系统的波束跳变与覆盖控制联合优化算法[J]. 通信学报, 2023, 44(4): 78-86.

Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system[J]. Journal on Communications, 2023, 44(4): 78-86.

图/表 10

图1

图2

图3

表1

图4

图5

图6

表2

图7

图8

参考文献 16

[1]	张晨, 张更新, 王显煜 . 基于跳波束的新一代高通量卫星通信系统设计[J]. 通信学报, 2020,41(7): 59-72.
	ZHANG C , ZHANG G X , WANG X Y . Design of next generation high throughput satellite communication system based on beam-hopping[J]. Journal on Communications, 2020,41(7): 59-72.
[2]	LEI L , LAGUNAS E , YUAN Y X ,et al. Deep learning for beam hopping in multibeam satellite systems[C]// Proceedings of 2020 IEEE 91st Vehicular Technology Conference. Piscataway:IEEE Press, 2020: 1-5.
[3]	LEI J , VáZQUEZ-CASTRO M á , . Multibeam satellite frequency/time duality study and capacity optimization[J]. Journal of Communications and Networks, 2011,13(5): 472-480.
[4]	CHOI J P , CHAN V W S . Optimum power and beam allocation based on traffic demands and channel conditions over satellite downlinks[J]. IEEE Transactions on Wireless Communications, 2005,4(6): 2983-2993.
[5]	WANG L B , HU X , MA S J ,et al. Dynamic beam hopping of multi-beam satellite based on genetic algorithm[C]// Proceedings of 2020 IEEE International Conference on Parallel ＆ Distributed Processing with Applications,Big Data ＆ Cloud Computing,Sustainable Computing ＆ Communications,Social Computing ＆ Networking. Piscataway:IEEE Press, 2021: 1364-1370.
[6]	TIAN F , HUANG L L , LIANG G ,et al. An efficient resource allocation mechanism for beam-hopping based LEO satellite communication system[C]// Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. Piscataway:IEEE Press, 2019: 1-5.
[7]	SHI D Y , LIU F , ZHANG T . Resource allocation in beam hopping communication satellite system[C]// Proceedings of International Wireless Communications and Mobile Computing. Piscataway:IEEE Press, 2020: 280-284.
[8]	WANG A Y , LEI L , LAGUNAS E ,et al. Joint beam-hopping scheduling and power allocation in NOMA-assisted satellite systems[C]// Proceedings of IEEE Wireless Communications and Networking Conference. Piscataway:IEEE Press, 2021: 1-6.
[9]	TANG J Y , BIAN D M , LI G X ,et al. Optimization method of dynamic beam position for LEO beam-hopping satellite communication systems[J]. IEEE Access, 2021,9: 57578-57588.
[10]	RAN Y Y , ZHOU X , HU H ,et al. Optimizing data centre energy efficiency via event driven deep reinforcement learning[C]// Proceedings of IEEE World Congress on Services. Piscataway:IEEE Press, 2022:20.
[11]	RAN Y Y , HU H , WEN Y G ,et al. Optimizing energy efficiency for data center via parameterized deep reinforcement learning[J]. IEEE Transactions on Services Computing, 2022,PP(99): 1-14.
[12]	HU X , LIU S J , WANG Y P ,et al. Deep reinforcement learning-based beam hopping algorithm in multibeam satellite systems[J]. IET Communications, 2019,13(16): 2485-2491.
[13]	LIU S J , HU X , WANG W D . Deep reinforcement learning based dynamic channel allocation algorithm in multibeam satellite systems[J]. IEEE Access, 2018,6: 15733-15742.
[14]	LUIS J J G , GUERSTER M , DEL PORTILLO I ,et al. Deep reinforcement learning for continuous power allocation in flexible high throughput satellites[C]// Proceedings of IEEE Cognitive Communications for Aerospace Applications Workshop. Piscataway:IEEE Press, 2019: 1-4.
[15]	LIAO X L , HU X , LIU Z J ,et al. Distributed intelligence:a verification for multi-agent DRL-based multibeam satellite resource allocation[J]. IEEE Communications Letters, 2020,24(12): 2785-2789.
[16]	LIN Z Y , NI Z Y , KUANG L L ,et al. Dynamic beam pattern and bandwidth allocation based on multi-agent deep reinforcement learning for beam hopping satellite systems[J]. IEEE Transactions on Vehicular Technology, 2022,71(4): 3917-3930.

仿真参数	取值
卫星高度/km	35 786
Ka 波段频率/GHz	20
总带宽B_tot/MHz	500
波束个数K/个	7
卫星总功率P_tot/dBW	34.5
波束发射功率P_k/dBW	26
小区数N /个	30
最大发射天线增益G_m/dBi	40.3
自由空间损耗L_f /dB	209.6
最大接收天线增益G_r/dBi	31.6
最小波束覆盖半径/m	500
最大波束覆盖半径/m	1 000
时隙持续时间/ms	2
重放缓存区大小	100 000
采样批量	128
折扣因子γ	0.9

方案	波束传输容量	波束覆盖小区数/个
方案1	F (1)	1
方案2	0.85F(1)	3
方案3	0.65F(1)	5

面向多波束卫星系统的波束跳变与覆盖控制联合优化算法

Joint beam hopping and coverage control optimization algorithm for multibeam satellite system

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 16

相关文章 15

Metrics

推荐阅读 0

[1]	金彪, 李逸康, 姚志强, 陈瑜霖, 熊金波. GenFedRL：面向深度强化学习智能体的通用联邦强化学习框架[J]. 通信学报, 2023, 44(6): 183-197.
[2]	李元诚, 秦永泰. 基于深度强化学习的软件定义安全中台QoS实时优化算法[J]. 通信学报, 2023, 44(5): 181-192.
[3]	沙宗轩, 霍如, 孙闯, 汪硕, 黄韬. 基于深度强化学习的转发效能感知流量调度算法[J]. 通信学报, 2022, 43(8): 30-40.
[4]	张宇, 程旻. NDN中边缘计算与缓存的联合优化[J]. 通信学报, 2022, 43(8): 164-175.
[5]	张先超, 赵耀, 叶海军, 樊锐. 无线网络多用户干扰下智能发射功率控制算法[J]. 通信学报, 2022, 43(2): 15-21.
[6]	苏新, 孟蕾蕾, 周一青, CELIMUGE Wu. 基于深度强化学习的海洋移动边缘计算卸载方法[J]. 通信学报, 2022, 43(10): 133-145.
[7]	杜丽娜, 卓力, 杨硕, 李嘉锋, 张菁. 基于强化学习的移动视频流业务码率自适应算法研究进展[J]. 通信学报, 2021, 42(9): 205-217.
[8]	贺文晨, 郭少勇, 邱雪松, 陈连栋, 张素香. 基于DRL的联邦学习节点选择方法[J]. 通信学报, 2021, 42(6): 62-71.
[9]	孙国林,欧睿杰,刘贵松. 基于深度强化学习的应急物联网切片资源预留算法[J]. 通信学报, 2020, 41(9): 8-20.
[10]	梁应敞,谭俊杰,Dusit Niyato. 智能无线通信技术研究概况[J]. 通信学报, 2020, 41(7): 1-17.
[11]	刘婷婷,罗义南,杨晨阳. 基于多智能体深度强化学习的分布式干扰协调[J]. 通信学报, 2020, 41(7): 38-48.
[12]	张沛,刘帅军,马治国,王晓晖,宋俊德. 基于深度增强学习和多目标优化改进的卫星资源分配算法[J]. 通信学报, 2020, 41(6): 51-60.
[13]	刘全,姜玉斌,胡智慧. 基于重要性采样的优势估计器[J]. 通信学报, 2019, 40(5): 108-116.
[14]	廖晓闽,严少虎,石嘉,谭震宇,赵钟灵,李赞. 基于深度强化学习的蜂窝网资源分配算法[J]. 通信学报, 2019, 40(2): 11-18.
[15]	兰巨龙,张学帅,胡宇翔,孙鹏浩. 基于深度强化学习的软件定义网络QoS优化[J]. 通信学报, 2019, 40(12): 60-67.