卫星CDN中基于DQN的资源编排算法

doi:10.11959/j.issn.2096-8930.2022042

Abstract

Abstract:

With the rapid development of space and information fi eld, hot content distribution intensive scenes will become one of the key directions of satellite network application, and satellite content delivery network (CDN) network is an important means to improve the effi ciency of air and space content distribution.In the architecture of satellite CDN network, due to the uneven time and space of business requirements, the scarcity of satellite resources and the insuffi cient adaptability of existing scheduling algorithms, scheduling algorithms for satellite resources are faced with problems such as high resource dimension, many computing states and large amount of computation, which will reduce the accuracy, response speed and computing performance of scheduling decisions.To solve this problem, a resource scheduling algorithm based on Deep Q-Learning (DQN) algorithm was proposed to improved the effi ciency and accuracy of satellite resource scheduling, and intelligently and quickly perceived the resource situation to make scheduling decisions.Firstly, the user requests were classifi ed, and the shortest path set that the satellite could communicated with was calculated according to the time-varying trajectory of the satellite and the resources of the satellite and the ground.After that, the related information of satellites and users was quantifi ed by Markov model modeling, and the optimal CDN storage node of satellites was calculated by DQN algorithm, which achieved the eff ects of reduced user request delay, reduced satellite-ground resource occupancy rate and improved cache hit rate.

Key words: satellite CDN, DQN, resource arrangement

CLC Number:

TN393

Jiaran ZHANG, Yating YANG, Tian SONG. Resource Scheduling Algorithm Based on DQN in Satellite CDN[J]. Space-Integrated-Ground Information Networks, 2022, 3(4): 45-54.

Figures/Tables 7

References 26

[1]	WAN P , YAN J , CAO Z G ,et al. Dynamic Source Routing algorithm in low-earth orbit Satellite Constellation[C]// Proceedings of 2006 International Conference on Communication Technology. Piscataway:IEEE Press, 2006: 1-4.
[2]	陈乃辉 . 卫星网络环境下数据一致性研究[D]. 武汉:华中科技大学, 2018.
	CHEN N H . Research on data consistency in satellite network environment[D]. Wuhan:Huazhong University of Science and Technology, 2018.
[3]	肖华 . CDN质量发展情况与优化方向探讨[J]. 广播电视网络, 2022,29(2): 87-89.
	XIAO H . Discussion on CDN quality development and optimization direction[J]. Radio ＆ Television Network, 2022,29(2): 87-89.
[4]	张美蓉, 镐梦婷, 王闯 ,等. 高低轨卫星异构网络资源管控策略与技术研究[J]. 天地一体化信息网络, 2021,2(4): 67-74.
	ZHANG M R , GAO M T , WANG C ,et al. Research on strategies and technologies for resource management and control of heterogeneous network of high and low orbit satellites[J]. Space-Integrated-Ground Information Networks, 2021,2(4): 67-74.
[5]	HAN C , LIU A J , HUO L Y ,et al. A prediction-based resource matching scheme for rentable LEO satellite communication network[J]. IEEE Communications Letters, 2020,24(2): 414-417.
[6]	唐宏, 陈戈, 陈步华 ,等. 内容分发网络原理与实践[J]. 电信科学, 2018,34(10): 181.
	TANG H , CHEN G , CHEN B H ,et al. Principle and practice of CDN[J]. Telecommunications Science, 2018,34(10): 181.
[7]	张焘 . 时变网络流分析与调度方法研究[D]. 西安:西安电子科技大学, 2020.
	ZHANG T . Network flow analysis and scheduling in time-varying netwoks[D]. Xi'an:Xidian University, 2020.
[8]	曹洋, 朱玉峰, 吕俊锋 ,等. 面向天地一体化内容分发业务的网内缓存机制研究[J]. 天地一体化信息网络, 2020,1(2): 48-56.
	CAO Y , ZHU Y F , LYU J F ,et al. Research on in-network caching mechanisms for space-integrated-ground content delivery service[J]. Space-Integrated-Ground Information Networks, 2020,1(2): 48-56.
[9]	LI Y , WANG Y , ZHANG Q Y ,et al. TCDS:a time-relevant graph based topology control in triple-layer satellite networks[J]. IEEE Wireless Communications Letters, 2020,9(3): 424-428.
[10]	吕俊锋 . 面向天地一体化网络内容分发的网内缓存机制研究[D]. 武汉:华中科技大学, 2020.
	LYU J F . In-network caching mechanisms for content delivery in space-ground integration network[D]. Wuhan:Huazhong University of Science and Technology, 2020.
[11]	WANG P , ZHANG X S , ZHANG S ,et al. Time-expanded graphbased resource allocation over the satellite networks[J]. IEEE Wireless Communications Letters, 2019,8(2): 360-363.
[12]	徐媚琳, 贾敏, 郭庆 . 基于SDN/NFV的卫星互联网服务功能资源分配研究[J]. 天地一体化信息网络, 2022,3(1): 44-49.
	XU M L , JIA M , GUO Q . Research on resource allocation of satellite Internet service function based on SDN/NFV[J]. Space-IntegratedGround Information Networks, 2022,3(1): 44-49.
[13]	李红艳, 张焘, 张靖乾 ,等. 基于时变图的天地一体化网络时间确定性路由算法与协议[J]. 通信学报, 2020,41(10): 116-129.
	LI H Y , ZHANG T , ZHANG J Q ,et al. Time deterministic routing algorithm and protocol based on time-varying graph over the spaceground integrated network[J]. Journal on Communications, 2020,41(10): 116-129.
[14]	ZHOU D , SHENG M , WANG Y X ,et al. Machine learning-based resource allocation in satellite networks supporting Internet of remote things[J]. IEEE Transactions on Wireless Communications, 2021,20(10): 6606-6621.
[15]	DUN C Y , TAN S H , JIN F L ,et al. Traffic allocation strategy based on reinforcement learning in two-layer satellite network[C]// Proceedings of 2021 International Conference on Internet,Education and Information Technology (IEIT). Piscataway:IEEE Press, 2021: 159-163.
[16]	CEVALLOS M J F , SATTLER R , CAULIER CISTERNA R P ,et al. Online service function chain deployment for live-streaming in virtualized content delivery networks:a deep reinforcement learning approach[J]. Future Internet, 2021,13(11): 278.
[17]	CHEN Z Y , HU J , MIN G Y ,et al. Adaptive and efficient resource allocation in cloud datacenters using actor-critic deep reinforcement learning[J]. IEEE Transactions on Parallel and Distributed Systems, 2022,33(8): 1911-1923.
[18]	LIU Y J , LU D J , ZHANG G J ,et al. Q-Learning based content placement method for dynamic cloud content delivery networks[J]. IEEE Access, 7: 66384-66394.
[19]	HUANG W Y , SONG T , AN J P . QA2:QoS-guaranteed access assistance for space–air–ground Internet of vehicle networks[J]. IEEE Internet of Things Journal, 2022,9(8): 5684-5695.
[20]	薛乃阳, 丁丹, 贾玉童 ,等. 基于DQN的异构测控资源联合调度方法[J]. 系统仿真学报, 2022: 1-13.
	XUE N Y , DING D , JIA Y T ,et al. Joint scheduling method of heterogeneous measurement and control resources based on DQN[J]. Journal of System Simulation, 2022: 1-13.
[21]	李贺武, 吴茜, 徐恪 ,等. 天地一体化网络研究进展与趋势[J]. 科技导报, 2016,34(14): 95-106.
	LI H W , WU Q , XU K ,et al. Progress and tendency of space and earth integrated network[J]. Science ＆ Technology Review, 2016,34(14): 95-106.
[22]	王凯东, 易克初, 田斌 . 低轨道(LEO)宽带卫星网络最短路由优化算法[J]. 中国科学E辑:信息科学, 2005,35(8): 850-874.
	WANG K D , YI K C , TIAN B . Shortest route optimization algorithm for low orbit (LEO) broadband satellite network[J]. Science in China,SerE, 2005,35(8): 850-874.
[23]	倪爽 . 星地一体化网络接入与存储资源协同管控技术研究[D]. 西安:西安电子科技大学, 2021.
	NI S . Coordinated access and cache resource management technology in terrestrial-satellite integrated network[D]. Xi'an:Xidian University, 2021.
[24]	徐建中 . 云计算环境下资源分配技术的研究[D]. 南京:南京理工大学, 2018.
	XU J Z . Research on resource allocation technology in cloud computing environment[D]. Nanjing:Nanjing University of Science and Technology, 2018.
[25]	曾首元 . 基于蚁群优化的低轨卫星网络负载均衡路由算法研究[D]. 广州:广州大学, 2022.
	ZENG S Y . Research on load balancing routing algorithm of LEO satellite network based on ant colony optimization[D]. Guangzhou:Guangzhou University, 2022.
[26]	杜琴 . 基于卫星切片网络的计算资源和无线资源联合分配机制的研究[D]. 北京:北京邮电大学, 2020.
	DU Q . Research on joint allocation mechanism of computing resource and wireless resource based on satellite slicing network[D]. Beijing:Beijing University of Posts and Telecommunications, 2020.

Metrics

Recommended 0

No Suggested Reading articles found!

算法类型	优点	缺点
资源不编排	计算简单，有效解决简单网络下单一需求问题	在复杂网络或多元需求的情况下，计算性能差，资源浪费高
动态规划	在简单网络多元业务的情况下，计算简单、多维度数据智能联合决策快速准确、支持动态决策	计算能力不足，难以支撑高维度、大数据量计算
深度学习	支持高维度、大数据量计算，计算准确率高	缺乏智能决策能力，难以解决多维度数据关系复杂的动态变化问题
强化学习	支持多维度数据智能联合决策，具有高适应性和低复杂性，解决动态网络变化问题	计算能力不足，数据规模受限
深度强化学习	支持高维度、大数据量计算，支持动态网络、多元业务和多维度数据智能联合决策，具有高适应性和高准确性	较高的算法成本，仅适合复杂网络、多元业务联合计算问题

Resource Scheduling Algorithm Based on DQN in Satellite CDN

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 26

Related Articles 15

Metrics

Recommended 0

算法名称	缓存命中率	传输开销	服务时延	资源利用率	算法成本
全部预缓存	高	高	低	低	低
贪心算法^[24]	中	中	中	中	低
蚁群算法^[25]	低	高	低	中	高
Q学习^[26]	高	中	中	中	中
DQN	高	低	低	高	高

[1]	Zihao PAN, Chen XIE, Heng WANG, Daoxing GUO. Research Status of CPM for Satellite-Terrestrial Integrated Network [J]. Space-Integrated-Ground Information Networks, 2023, 4(1): 42-49.
[2]	Yingzhe DOU, Guanchang XUE, Mingchuan YANG. High-Throughput Satellite Precoding Algorithm Based on Non-Ideal Channel State Information [J]. Space-Integrated-Ground Information Networks, 2023, 4(1): 66-72.
[3]	Zhiyong LUO, Ning XIN, Shaohui SUN. Beamforming Technology of LEO Satellite in Satellite-Terrestrial Integrated Network [J]. Space-Integrated-Ground Information Networks, 2022, 3(4): 22-30.
[4]	Zhen WU, Jingyu YANG, Shuo WANG, Zhenghuan WANG, Min JIA. Design and Implementation of Intelligent Anti-Interference for Data Link [J]. Space-Integrated-Ground Information Networks, 2022, 3(4): 31-44.
[5]	Jianxiang CHU, Yin HUANG, Chong WANG, Yunze CAO. Multi-Users Dynamic Bandwidth Allocation Technology of SDTN [J]. Space-Integrated-Ground Information Networks, 2022, 3(4): 83-91.
[6]	Runxin ZHANG, Shun ZHANG, Jianpeng MA, Jing ZHANG, Yunfei WU. Research on HARQ Technology in Low-Orbit Satellite Communication Network [J]. Space-Integrated-Ground Information Networks, 2022, 3(4): 92-100.
[7]	Hui LI. From Frequency Spectrum Occupation to Effi ciently Utilization [J]. Space-Integrated-Ground Information Networks, 2021, 2(1): 46-51.
[8]	Yongjian LIU, Bincheng LI, Yanzhe WANG. Key Issues in Development and Application of the Sky-Based Internet of Things [J]. Space-Integrated-Ground Information Networks, 2021, 2(1): 81-86.
[9]	Xiaoning ZHANG, Lidong ZHU. Communication and Security Integrated Space-Earth Heterogeneous Network Architecture [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 11-16.
[10]	Wenlan DIAO, Jianping AN, Yu ZHANG, Han HU. Multiple Addressing Coordination and Switching Technology in Space-Integrated-Ground Information Network [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 42-47.
[11]	Yang CAO, Yufeng ZHU, Junfeng LV, Tao JIANG, Peng GAO, Dongchen ZHANG. Research on In-Network Caching Mechanisms for Space-Integrated-Ground Content Delivery Service [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 48-56.
[12]	Guohua WEI, Hui LI, Yongjie BAI, Gengxin LI, Kaixuan XING. Space-Terrestrial Integrated Multi-Identifier Network with Endogenous Security [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 66-72.
[13]	Huibing WANG, Zuoya ZHENG, Sida OUYANG, Yue ZHANG, Chao YANG, Yu HUANG. Application Analysis of Space-Based Relay in Land Resources Satellite Image Acquisition [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 103-108.
[14]	Yangyang LU, Guangyue XUE, Hanchang SUN, Qingxia XIE. Design of Multimodal Transport Information Service System Based on Space-Based Internet of Things [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 116-120.
[15]	Qiao HU, Zuoya ZHENG, Gang LIU, Linyao QIU, Yanfeng WU. Discussion on Disaster Remote Sensing Application Service Based on Ground Information Port [J]. Space-Integrated-Ground Information Networks, 2020, 1(2): 121-127.