天地算力网络中的异构资源协同博弈

doi:10.11959/j.issn.1000-436x.2023214

Abstract

Abstract:

To deal with the resource competition among satellites in the multi-satellite space-ground computing network, a space-ground heterogeneous resource cooperative game mechanism was designed in terms of the computing and spectrum domains.Each satellite published a computing task which was independent of other tasks and relied on UE to generate raw data.By competing the resources of user terminals and UE, the task offloading and processing was achieved.To provide real-time data services, a distributed scheme was proposed based on multi-agent reinforcement learning to coordinate the computing and spectrum resource competition among satellites, thereby minimizing the system latency.Simulation results indicated that, compared with the existing schemes, the proposed algorithm achieves a lower system latency by fully utilizing the computing and spectrum resources and coordinating the resource competition.

Key words: space-ground computing power network, heterogeneous resource cooperative game, multi-agent reinforcement learning

CLC Number:

TN927

Yutong ZHANG, Yuming PENG, Boya DI, Lingyang SONG. Heterogeneous resource cooperative game in space-ground computing power network[J]. Journal on Communications, 2023, 44(12): 15-27.

Figures/Tables 16

符号	定义
L	卫星总数
N	TST总数
M	每个TST下的UE总数
l^n,m	UE m-TST n-卫星l链路上的子任务
$λ_{UE}^{n, m} (l)$	子任务l^n,m的数据产生速率
$x_{TST}^{n, m} (l)$	对于子任务l^n,m，在TST n上计算的原始数据占其接收到的原始数据总数的百分比
$x_{UE}^{n, m} (l)$	对于子任务l^n,m，在UE m上计算的原始数据占其产生的原始数据总数的百分比
$ϕ_{TST}^{n, m} (l)$	子任务l^n,m的卫星-TST传输资源
$ϕ_{UE}^{n, m} (l)$	子任务l^n,m的TST-UE传输资源
$θ_{SAT}^{n, m} (l)$	子任务l^n,m的卫星计算资源
$θ_{TST}^{n, m} (l)$	子任务l^n,m的TST计算资源
$θ_{UE}^{n, m} (l)$	子任务l^n,m的UE计算资源
$t_{SAT}^{n, m} (l)$	子任务l^n,m在卫星上的数据计算时间
$t_{TST}^{n, m} (l)$	子任务l^n,m在TST上的数据计算时间
$t_{UE}^{n, m} (l)$	子任务l^n,m在UE上的数据计算时间
$τ_{TST}^{n, m} (l)$	子任务l^n,m从TST到卫星的数据传输时间
$τ_{UE}^{n, m} (l)$	子任务l^n,m从UE到TST的数据传输时间
$T^{n, m} (l)$	子任务l^n,m的时延
T (l)	任务l的时延
T	系统时延
ρ	数据压缩率

缩减原则	占 $\| A \|$ 的比例
x_UE=1且x_TST≠0	$\frac{1}{d} (1 - \frac{1}{d})$
x_UE=0且θ_UE≠0	$\frac{1}{d} (1 - \frac{1}{c})$
x_TST=0且θ_TST≠0	$\frac{1}{d} (1 - \frac{1}{c})$
x_TST=1且θ_SAT≠0	$\frac{1}{d} (1 - \frac{1}{c})$

重叠部分	占 $\| A \|$ 的比例
x_UE=1，x_TST=1且θ_SAT≠0	$\frac{1}{d} \frac{1}{d} (1 - \frac{1}{c})$
x_UE=0，x_TST=0，θ_UE≠0且θ_TST≠0	${[\frac{1}{d} (1 - \frac{1}{c})]}^{2}$
x_UE=0，x_TST=1，θ_UE=0且θ_TST≠0	${[\frac{1}{d} (1 - \frac{1}{c})]}^{2}$

References 30

[1]	DI B Y , ZHANG H L , SONG L Y ,et al. Ultra-dense LEO:integrating terrestrial-satellite networks into 5G and beyond for data offloading[J]. IEEE Transactions on Wireless Communications, 2019,18(1): 47-62.
[2]	SU Y T , LIU Y Q , ZHOU Y Q ,et al. Broadband LEO satellite communications:architectures and key technologies[J]. IEEE Wireless Communications, 2019,26(2): 55-61.
[3]	胡馨元, 邓若琪, 邸博雅 ,等. 可重构全息超表面辅助卫星通信关键技术[J]. 电信科学, 2022,38(10): 46-56.
	HU X Y , DENG R Q , DI B Y ,et al. Key technologies of satellite communications aided by reconfigurable holographic surfaces[J]. Telecommunications Science, 2022,38(10): 46-56.
[4]	VOIGT S , GIULIO-TONOLO F , LYONS J ,et al. Global trends in satellite-based emergency mapping[J]. Science, 2016,353(6296): 247-252.
[5]	王鹏飞, 邸博雅, 唐斌 ,等. 面向广域海洋覆盖的密集低轨卫星星座[J]. 无线电通信技术, 2021,47(4): 402-409.
	WANG P F , DI B Y , TANG B ,et al. Ultra?dense LEO satellite constellation design for wide ocean coverage area[J]. Radio Communications Technology, 2021,47(4): 402-409.
[6]	ZHOU Y , LIU L , WANG L ,et al. Service-aware 6G:an intelligent and open network based on the convergence of communication,computing and caching[J]. Digital Communications and Networks, 2020,6(3): 253-260.
[7]	ZHOU Y Q , TIAN L , LIU L ,et al. Fog computing enabled future mobile communication networks:a convergence of communication and computing[J]. IEEE Communications Magazine, 2019,57(5): 20-27.
[8]	LIAO X L , HU X , LIU Z J ,et al. Distributed intelligence:a verification for multi-agent DRL-based multibeam satellite resource allocation[J]. IEEE Communications Letters, 2020,24(12): 2785-2789.
[9]	LIU R , GUO K F , AN K ,et al. Resource allocation for NOMA-enabled cognitive satellite-UAV-terrestrial networks with imperfect CSI[J]. IEEE Transactions on Cognitive Communications and Networking, 2023,9(4): 963-976.
[10]	WANG W L , WEI J , ZHAO S H ,et al. Energy efficiency resource allocation based on spectrum-power tradeoff in distributed satellite cluster network[J]. Wireless Networks, 2020,26(6): 4389-4402.
[11]	ZHANG Y D , YIN L G , JIANG C X ,et al. Joint beamforming design and resource allocation for terrestrial-satellite cooperation system[J]. IEEE Transactions on Communications, 2019,68(2): 778-791.
[12]	ZHANG S H , CUI G F , LONG Y T ,et al. Joint computing and communication resource allocation for satellite communication networks with edge computing[J]. China Communications, 2021,18(7): 236-252.
[13]	FU S , GAO J , ZHAO L . Collaborative multi-resource allocation in terrestrial-satellite network towards 6G[J]. IEEE Transactions on Wireless Communications, 2021,20(11): 7057-7071.
[14]	DING C F , WANG J B , CHENG M ,et al. Dynamic transmission and computation resource optimization for dense LEO satellite assisted mobile-edge computing[J]. IEEE Transactions on Communications, 2023,71(5): 3087-3102.
[15]	DENG R Q , DI B Y , CHEN S Z ,et al. Ultra-dense LEO satellite offloading for terrestrial networks:how much to pay the satellite operator?[J]. IEEE Transactions on Wireless Communications, 2020,19(10): 6240-6254.
[16]	东方星 . 我国高分卫星与应用简析[J]. 卫星应用, 2015(3): 44-48.
	DONGFANG X . Analysis of high score satellite and its application in China[J]. Satellite Application, 2015(3): 44-48.
[17]	白徐祥, 官山林 . 动中通卫星通信天线[J]. 无线通信技术, 2004,13(1): 26-28,32.
	BAI X X , GUAN S L . Antenna design for satellite communication in motion[J]. Wireless Communication Technology, 2004,13(1): 26-28,32.
[18]	GAO X Q , LIU R K , KAUSHIK A ,et al. Dynamic resource allocation for virtual network function placement in satellite edge clouds[J]. IEEE Transactions on Network Science and Engineering, 2022,9(4): 2252-2265.
[19]	REN J K , YU G D , CAI Y L ,et al. Latency optimization for resource allocation in mobile-edge computation offloading[J]. IEEE Transactions on Wireless Communications, 2018,17(8): 5506-5519.
[20]	ZHANG Y T , DI B Y , ZHENG Z J ,et al. Distributed multi-cloud multi-access edge computing by multi-agent reinforcement learning[J]. IEEE Transactions on Wireless Communications, 2021,20(4): 2565-2578.
[21]	ZHAO N , YE Z Y , PEI Y Y ,et al. Multi-agent deep reinforcement learning for task offloading in UAV-assisted mobile edge computing[J]. IEEE Transactions on Wireless Communications, 2022,21(9): 6949-6960.
[22]	XU J , CHEN L X , REN S L . Online learning for offloading and autoscaling in energy harvesting mobile edge computing[J]. IEEE Transactions on Cognitive Communications and Networking, 2017,3(3): 361-373.
[23]	CLAUS C , BOUTILIER C . The dynamics of reinforcement learning in cooperative multiagent systems[C]// Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence. New York:ACM Press, 1998: 746-752.
[24]	TEYMOORI P , BOUKERCHE A . Dynamic multi-user computation offloading for mobile edge computing using game theory and deep reinforcement learning[C]// Proceedings of IEEE International Conference on Communications. Piscataway:IEEE Press, 2022: 1930-1935.
[25]	GOMES E R , KOWALCZYK R . Dynamic analysis of multiagent Q-learning with ε-greedy exploration[C]// Proceedings of the 26th Annual International Conference on Machine Learning. New York:ACM Press, 2009: 369-376.
[26]	SADHU A K , KONAR A . An efficient computing of correlated equilibrium for cooperative Q-learning-based multi-robot planning[J]. IEEE Transactions on Systems,Man,and Cybernetics:Systems, 2020,50(8): 2779-2794.
[27]	JU Y , CHEN Y C , CAO Z W ,et al. Joint secure offloading and resource allocation for vehicular edge computing network:a multi-agent deep reinforcement learning approach[J]. IEEE Transactions on Intelligent Transportation Systems, 2023,24(5): 5555-5569.
[28]	WAQAR N , HASSAN S A , MAHMOOD A ,et al. Computation offloading and resource allocation in MEC-enabled integrated aerial-terrestrial vehicular networks:a reinforcement learning approach[J]. IEEE Transactions on Intelligent Transportation Systems, 2022,23(11): 21478-21491.
[29]	DENG R Q , DI B Y , ZHANG H L ,et al. Ultra-dense LEO satellite constellations:how many LEO satellites do we need[J]. IEEE Transactions on Wireless Communications, 2021,20(8): 4843-4857.
[30]	WANG P F , DI B Y , SONG L Y . Mega-constellation design for integrated satellite-terrestrial networks for global seamless connectivity[J]. IEEE Wireless Communications Letters, 2022,11(8): 1669-1673.

Metrics

Recommended 0

No Suggested Reading articles found!

参数	值
每UE CPU频率/(cycle·s^-1)	3×10¹
每TST CPU频率/(cycle·s^-1)	8×10¹⁰
每卫星 CPU频率/(cycle·s^-1)	2.4×10¹⁰
处理每比特数据所需CPU周期/(cycle·bit^-1)	100
TST-UE链路带宽/MHz	20
TST-UE链路子信道数量	500
TST-UE链路发射功率/dBm	43
TST-UE链路发射天线增益/dBi	31.2
TST-UE链路噪声/(dBm·Hz^-1)	-120
卫星-TST链路带宽/MHz	100
卫星-TST链路子信道数量	16
卫星-TST链路发射功率/W	2
卫星-TST链路发射天线增益/dBi	37.1
卫星-TST链路噪声/(dBm·Hz^-1)	-174
数据压缩率	10%
折扣率	0.9
初始学习率α₀	0.15
初始探索率ε₀	0.1

卫星数量/颗	中心式算法/s	SHRCG/s	卫星计算/s	本地计算/s
1	1.171 8	1.178 5	1.210 0	1.502 6
2	1.336 4	1.339 3	1.410 0	2.004 6
3	1.492 6	1.503 5	1.610 0	2.506 6

TST数量/个	中心式算法/s	SHRCG/s	卫星计算/s	本地计算/s
1	1.436 4	1.439 3	1.510 0	2.104 6
2	1.443 6	1.451 0	1.520 0	2.105 2
3	1.450 7	1.459 3	1.530 0	2.105 8

UE数量/个	中心式算法/s	SHRCG/s	卫星计算/s	本地计算/s
1	1.336 4	1.339 3	1.410 0	2.004 6
2	1.554 6	1.574 4	1.820 0	2.009 2
3	1.663 9	1.669 6	2.230 0	2.013 8

设备数量	卫星/次	TST/次	UE/次
1	3.1×10⁴	4.9×10⁴	4.9×10⁴
2	4.9×10⁴	5.3×10⁴	5.8×10⁴
3	5.3×10⁵	6.1×10⁴	5.5×10⁴

Heterogeneous resource cooperative game in space-ground computing power network

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 30

Related Articles 3

Metrics

Recommended 0

[1]	Feng YAN, Xiaowei LIN, Zhenghao LI, Xia XU, Weiwei XIA, Lianfeng SHEN. Spectrum allocation algorithm based on multi-agent reinforcement learning in smart grid [J]. Journal on Communications, 2023, 44(9): 12-24.
[2]	Bingyi LIU, Yuhao LIU, Weizhen HAN, Zhenchang XIA, Libing WU, Shengwu XIONG. Edge intelligence-assisted routing protocol for Internet of vehicles via reinforcement learning [J]. Journal on Communications, 2023, 44(11): 110-119.
[3]	Wenjun XU, Silei WU, Fengyu WANG, Lan LIN, Guojun LI, Zhi ZHANG. Large-scale post-disaster user distributed coverage optimization based on multi-agent reinforcement learning [J]. Journal on Communications, 2022, 43(8): 1-16.