面向多波束卫星系统的波束跳变与覆盖控制联合优化算法

doi:10.11959/j.issn.1000-436x.2023076

Abstract

Abstract:

To improve the performance of multibeam satellite (MBS) systems, a deep reinforcement learning-based algorithm to jointly optimize the beam hopping and coverage control (BHCC) algorithm for MBS was proposed.Firstly, the resource allocation problem in MBS was transformed to a multi-objective optimization problem with the objective maximizing the system throughput and minimizing the packet loss rate of the MBS.Secondly, the MBS environment was characterized as a multi-dimensional matrix, and the objective problem was modelled as a Markov decision process considering stochastic communication requirements.Finally, the objective problem was solved by combining the powerful feature extraction and learning capabilities of deep reinforcement learning.In addition, a single-intelligence polling multiplexing mechanism was proposed to reduce the search space and convergence difficulty and accelerate the training of BHCC.Compared with the genetic algorithm, the simulation results show that BHCC improves the throughput of MBS and reduces the packet loss rate of the system, greedy algorithm, and random algorithm.Besides, BHCC performs better in different communication scenarios compared with a deep reinforcement learning algorithm, which do not consider the adaptive beam coverage.

Key words: multibeam satellite, deep reinforcement learning, beam hopping technology, beam coverage control

CLC Number:

TN92

Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system[J]. Journal on Communications, 2023, 44(4): 78-86.

Figures/Tables 10

References 16

[1]	张晨, 张更新, 王显煜 . 基于跳波束的新一代高通量卫星通信系统设计[J]. 通信学报, 2020,41(7): 59-72.
	ZHANG C , ZHANG G X , WANG X Y . Design of next generation high throughput satellite communication system based on beam-hopping[J]. Journal on Communications, 2020,41(7): 59-72.
[2]	LEI L , LAGUNAS E , YUAN Y X ,et al. Deep learning for beam hopping in multibeam satellite systems[C]// Proceedings of 2020 IEEE 91st Vehicular Technology Conference. Piscataway:IEEE Press, 2020: 1-5.
[3]	LEI J , VáZQUEZ-CASTRO M á , . Multibeam satellite frequency/time duality study and capacity optimization[J]. Journal of Communications and Networks, 2011,13(5): 472-480.
[4]	CHOI J P , CHAN V W S . Optimum power and beam allocation based on traffic demands and channel conditions over satellite downlinks[J]. IEEE Transactions on Wireless Communications, 2005,4(6): 2983-2993.
[5]	WANG L B , HU X , MA S J ,et al. Dynamic beam hopping of multi-beam satellite based on genetic algorithm[C]// Proceedings of 2020 IEEE International Conference on Parallel ＆ Distributed Processing with Applications,Big Data ＆ Cloud Computing,Sustainable Computing ＆ Communications,Social Computing ＆ Networking. Piscataway:IEEE Press, 2021: 1364-1370.
[6]	TIAN F , HUANG L L , LIANG G ,et al. An efficient resource allocation mechanism for beam-hopping based LEO satellite communication system[C]// Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. Piscataway:IEEE Press, 2019: 1-5.
[7]	SHI D Y , LIU F , ZHANG T . Resource allocation in beam hopping communication satellite system[C]// Proceedings of International Wireless Communications and Mobile Computing. Piscataway:IEEE Press, 2020: 280-284.
[8]	WANG A Y , LEI L , LAGUNAS E ,et al. Joint beam-hopping scheduling and power allocation in NOMA-assisted satellite systems[C]// Proceedings of IEEE Wireless Communications and Networking Conference. Piscataway:IEEE Press, 2021: 1-6.
[9]	TANG J Y , BIAN D M , LI G X ,et al. Optimization method of dynamic beam position for LEO beam-hopping satellite communication systems[J]. IEEE Access, 2021,9: 57578-57588.
[10]	RAN Y Y , ZHOU X , HU H ,et al. Optimizing data centre energy efficiency via event driven deep reinforcement learning[C]// Proceedings of IEEE World Congress on Services. Piscataway:IEEE Press, 2022:20.
[11]	RAN Y Y , HU H , WEN Y G ,et al. Optimizing energy efficiency for data center via parameterized deep reinforcement learning[J]. IEEE Transactions on Services Computing, 2022,PP(99): 1-14.
[12]	HU X , LIU S J , WANG Y P ,et al. Deep reinforcement learning-based beam hopping algorithm in multibeam satellite systems[J]. IET Communications, 2019,13(16): 2485-2491.
[13]	LIU S J , HU X , WANG W D . Deep reinforcement learning based dynamic channel allocation algorithm in multibeam satellite systems[J]. IEEE Access, 2018,6: 15733-15742.
[14]	LUIS J J G , GUERSTER M , DEL PORTILLO I ,et al. Deep reinforcement learning for continuous power allocation in flexible high throughput satellites[C]// Proceedings of IEEE Cognitive Communications for Aerospace Applications Workshop. Piscataway:IEEE Press, 2019: 1-4.
[15]	LIAO X L , HU X , LIU Z J ,et al. Distributed intelligence:a verification for multi-agent DRL-based multibeam satellite resource allocation[J]. IEEE Communications Letters, 2020,24(12): 2785-2789.
[16]	LIN Z Y , NI Z Y , KUANG L L ,et al. Dynamic beam pattern and bandwidth allocation based on multi-agent deep reinforcement learning for beam hopping satellite systems[J]. IEEE Transactions on Vehicular Technology, 2022,71(4): 3917-3930.

Metrics

Recommended 0

No Suggested Reading articles found!

仿真参数	取值
卫星高度/km	35 786
Ka 波段频率/GHz	20
总带宽B_tot/MHz	500
波束个数K/个	7
卫星总功率P_tot/dBW	34.5
波束发射功率P_k/dBW	26
小区数N /个	30
最大发射天线增益G_m/dBi	40.3
自由空间损耗L_f /dB	209.6
最大接收天线增益G_r/dBi	31.6
最小波束覆盖半径/m	500
最大波束覆盖半径/m	1 000
时隙持续时间/ms	2
重放缓存区大小	100 000
采样批量	128
折扣因子γ	0.9

方案	波束传输容量	波束覆盖小区数/个
方案1	F (1)	1
方案2	0.85F(1)	3
方案3	0.65F(1)	5

Joint beam hopping and coverage control optimization algorithm for multibeam satellite system

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 16

Related Articles 15

Metrics

Recommended 0

[1]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[2]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[3]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[4]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[5]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[6]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[7]	Li’na DU, Li ZHUO, Shuo YANG, Jiafeng LI, Jing ZHANG. Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services [J]. Journal on Communications, 2021, 42(9): 205-217.
[8]	Wenchen HE, Shaoyong GUO, Xuesong QIU, Liandong CHEN, Suxiang ZHANG. Node selection method in federated learning based on deep reinforcement learning [J]. Journal on Communications, 2021, 42(6): 62-71.
[9]	Guolin SUN,Ruijie OU,Guisong LIU. Deep reinforcement learning-based resource reservation algorithm for emergency Internet-of-things slice [J]. Journal on Communications, 2020, 41(9): 8-20.
[10]	Yingchang LIANG,Junjie TAN,Dusit Niyato. Overview on intelligent wireless communication technology [J]. Journal on Communications, 2020, 41(7): 1-17.
[11]	Tingting LIU,Yi’nan LUO,Chenyang YANG. Distributed interference coordination based on multi-agent deep reinforcement learning [J]. Journal on Communications, 2020, 41(7): 38-48.
[12]	Pei ZHANG,Shuaijun LIU,Zhiguo MA,Xiaohui WANG,Junde SONG. Improved satellite resource allocation algorithm based on DRL and MOP [J]. Journal on Communications, 2020, 41(6): 51-60.
[13]	Quan LIU,Yubin JIANG,Zhihui HU. Advantage estimator based on importance sampling [J]. Journal on Communications, 2019, 40(5): 108-116.
[14]	Xiaomin LIAO,Shaohu YAN,Jia SHI,Zhenyu TAN,Zhongling ZHAO,Zan LI. Deep reinforcement learning based resource allocation algorithm in cellular networks [J]. Journal on Communications, 2019, 40(2): 11-18.
[15]	Julong LAN,Xueshuai ZHANG,Yuxiang HU,Penghao SUN. Software-defined networking QoS optimization based on deep reinforcement learning [J]. Journal on Communications, 2019, 40(12): 60-67.