面向B5G多业务场景基于D3QN的双时间尺度网络切片算法

doi:10.11959/j.issn.1000-436x.2022207

通信学报 ›› 2022, Vol. 43 ›› Issue (11): 213-224.doi: 10.11959/j.issn.1000-436x.2022207

面向B5G多业务场景基于D3QN的双时间尺度网络切片算法

陈赓¹, 齐书虎¹, 沈斐², 曾庆田¹

¹ 山东科技大学电子信息工程学院，山东青岛 266590
² 中国科学院上海微系统与信息技术研究所，上海 200050

修回日期:2022-10-18 出版日期:2022-11-25 发布日期:2022-11-01
作者简介:陈赓（1984− ），男，山东潍坊人，博士，山东科技大学副教授、硕士生导师，主要研究方向为异构网络、泛在网络和软件定义移动网络、无线资源管理和优化算法
齐书虎（1998− ），男，山东聊城人，山东科技大学硕士生，主要研究方向为网络切片、资源分配
沈斐（1983− ），女，江苏南京人，博士，中国科学院上海微系统与信息技术研究所研究员、博士生导师，主要研究方向为无线通信、边缘计算和雾计算的资源优化
曾庆田（1976− ），男，山东潍坊人，博士，山东科技大学教授、博士生导师，主要研究方向为Petri网、过程挖掘和知识管理
基金资助:
国家自然科学基金资助项目(61701284);国家自然科学基金资助项目(61871370);国家重点研发计划基金资助项目(2019YFE0120700);国家重点研发计划基金资助项目(2019YFB1803101);山东省自然科学基金资助项目(ZR2022MF226);青岛市应用基础研究计划基金资助项目(19-6-2-1-cg);山东科技大学菁英计划基金资助项目(skr21-3-B-048);中国科学院百人计划基金资助项目(E06BRA1001);山东省泰山学者计划基金资助项目(ts20190936);山东省高校青年创新团队科技支撑计划基金资助项目(2019KJN024)

Dual time scale network slicing algorithm based on D3QN for B5G multi-service scenarios

Geng CHEN¹, Shuhu QI¹, Fei SHEN², Qingtian ZENG¹

¹ College of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, China
² Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai 200050, China

Revised:2022-10-18 Online:2022-11-25 Published:2022-11-01
Supported by:
The National Natural Science Foundation of China(61701284);The National Natural Science Foundation of China(61871370);The National Key Research and Development Program of China(2019YFE0120700);The National Key Research and Development Program of China(2019YFB1803101);The Natural Science Foundation of Shandong Province(ZR2022MF226);The Innovative Research Foundation of Qingdao(19-6-2-1-cg);The Elite Plan Project of Shandong University of Science and Technology(skr21-3-B-048);The Hundred Talent Program of Chinese Academy of Sciences(E06BRA1001);The Taishan Scholar Program of Shandong Province(ts20190936);The Science and Technology Support Plan of Youth Innovation Team of Shandong Higher School(2019KJN024)

摘要/Abstract

摘要：

为了有效满足不同切片的差异化服务质量需求，面向B5G多业务场景提出了一种基于竞争双深度Q网络（D3QN）的双时间尺度网络切片算法。研究了联合资源切片和调度问题，以归一化处理后的频谱效率和不同切片用户服务质量指数的加权和作为优化目标。在大时间尺度内，SDN控制器根据每种业务的资源需求利用D3QN算法预先分配给不同的切片，然后根据基站负载状态执行基站级资源更新。在小时间尺度内，基站通过轮询调度算法将资源调度到终端用户。仿真结果表明，所提算法在保证切片用户服务质量需求、频谱效率和系统效用方面具有优异的性能。与其他4种基准算法相比，所提算法的系统效用分别提升了3.22%、3.81%、7.48%和21.14%。

关键词: 双时间尺度, 资源分配, 网络切片, 竞争双深度Q网络, 系统效用

Abstract:

To effectively meet the differentiated quality of service (QoS) requirements of different slices, a dual time scale network slicing resource allocation algorithm based on dueling double DQN (D3QN) was proposed for B5G multi-service scenarios.The joint resource slicing and scheduling problem was formulated, with the weighted sum of the normalized spectral efficiency (SE) and the QoS of users indices of different slices as the optimization objective.On large time scale, the SDN controller used the D3QN algorithm to pre-allocate resources to different slices based on the resource requirements of each service, and then performed BS-level resource updating based on the load status of BS.On small time scale, the BS scheduled resources to end-users by using the round-robin scheduling algorithm.The simulation results show that the proposed algorithm has excellent performance in ensuring the QoS requirements of slice users, SE and system utility.Compared with the other 4 baseline algorithms, the system utility is improved by 3.22%, 3.81%, 7.48% and 21.14%, respectively.

Key words: dual time scale, resource allocation, network slicing, dueling double DQN, system utility

中图分类号:

TN929.5

陈赓, 齐书虎, 沈斐, 曾庆田. 面向B5G多业务场景基于D3QN的双时间尺度网络切片算法[J]. 通信学报, 2022, 43(11): 213-224.

Geng CHEN, Shuhu QI, Fei SHEN, Qingtian ZENG. Dual time scale network slicing algorithm based on D3QN for B5G multi-service scenarios[J]. Journal on Communications, 2022, 43(11): 213-224.

图/表 14

图1

图2

图3

表1

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

参考文献 24

[1]	DOGRA A , JHA R K , JAIN S . A survey on beyond 5G network with the advent of 6G:architecture and emerging technologies[J]. IEEE Access, 2021,9: 67512-67547.
[2]	MLIKA Z , CHERKAOUI S . Massive IoT access with NOMA in 5G networks and beyond using online competitiveness and learning[J]. IEEE Internet of Things Journal, 2021,8(17): 13624-13639.
[3]	ELAYOUBI S E , JEMAA S B , ALTMAN Z ,et al. 5G RAN slicing for verticals:enablers and challenges[J]. IEEE Communications Magazine, 2019,57(1): 28-34.
[4]	FILALI A , ABOUAOMAR A , CHERKAOUI S ,et al. Multi-access edge computing:a survey[J]. IEEE Access, 2020,8: 197017-197046.
[5]	SHU Z G , TALEB T . A novel QoS framework for network slicing in 5G and beyond networks based on SDN and NFV[J]. IEEE Network, 2020,34(3): 256-263.
[6]	HUA Y X , LI R P , ZHAO Z F ,et al. GAN-powered deep distributional reinforcement learning for resource management in network slicing[J]. IEEE Journal on Selected Areas in Communications, 2020,38(2): 334-349.
[7]	LI R P , WANG C J , ZHAO Z F ,et al. The LSTM-based advantage actor-critic learning for resource management in network slicing with user mobility[C]// Proceedings of IEEE Communications Letters. Piscataway:IEEE Press, 2020: 2005-2009.
[8]	FILALI A , MLIKA Z , CHERKAOUI S ,et al. Dynamic SDN-based radio access network slicing with deep reinforcement learning for URLLC and eMBB services[J]. IEEE Transactions on Network Science and Engineering, 2022,9(4): 2174-2187.
[9]	SUN G L , XIONG K , BOATENG G O ,et al. Autonomous resource provisioning and resource customization for mixed traffics in virtualized radio access network[J]. IEEE Systems Journal, 2019,13(3): 2454-2465.
[10]	SHAO Y , LI R P , HU B ,et al. Graph attention network-based multi-agent reinforcement learning for slicing resource management in dense cellular network[J]. IEEE Transactions on Vehicular Technology, 2021,70(10): 10792-10803.
[11]	ABIKO Y , SAITO T , IKEDA D ,et al. Flexible resource block allocation to multiple slices for radio access network slicing using deep reinforcement learning[J]. IEEE Access, 2020,8: 68183-68198.
[12]	YANG G , LIU Q , ZHOU X W ,et al. Two-tier resource allocation in dynamic network slicing paradigm with deep reinforcement learning[C]// Proceedings of 2019 IEEE Global Communications Conference. Piscataway:IEEE Press, 2019: 1-6.
[13]	WU H Q , CHEN J Y , ZHOU C H ,et al. Learning-based joint resource slicing and scheduling in space-terrestrial integrated vehicular networks[J]. Journal of Communications and Information Networks, 2021,6(3): 208-223.
[14]	SHI W S , LI J L , YANG P ,et al. Two-level soft RAN slicing for customized services in 5G-and-beyond wireless communications[J]. IEEE Transactions on Industrial Informatics, 2022,18(6): 4169-4179.
[15]	PAPA A , JANO A , AYVA?IK S ,et al. User-based quality of service aware multi-cell radio access network slicing[J]. IEEE Transactions on Network and Service Management, 2022,19(1): 756-768.
[16]	HU J Z , ZHENG Z J , DI B Y ,et al. Multi-layer radio network slicing for heterogeneous communication systems[J]. IEEE Transactions on Network Science and Engineering, 2020,7(4): 2378-2391.
[17]	SUN Y , QIN S , FENG G ,et al. Service provisioning framework for RAN slicing:user admissibility,slice association and bandwidth allocation[J]. IEEE Transactions on Mobile Computing, 2021,20(12): 3409-3422.
[18]	ZHAO N , LIANG Y C , NIYATO D ,et al. Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks[J]. IEEE Transactions on Wireless Communications, 2019,18(11): 5141-5152.
[19]	孙国林, 欧睿杰, 刘贵松 . 基于深度强化学习的应急物联网切片资源预留算法[J]. 通信学报, 2020,41(9): 8-20.
	SUN G L , OU R J , LIU G S . Deep reinforcement learning-based resource reservation algorithm for emergency Internet-of-things slice[J]. Journal on Communications, 2020,41(9): 8-20.
[20]	亓伟敬, 宋清洋, 郭磊 . 面向软件定义多模态车联网的双时间尺度RAN切片资源分配[J]. 通信学报, 2022,43(4): 60-70.
	QI W J , SONG Q Y , GUO L . Dual time scale resource allocation for RAN slicing in software-defined oriented polymorphic IoV[J]. Journal on Communications, 2022,43(4): 60-70.
[21]	喻鹏, 张俊也, 李文璟 ,等. 移动边缘网络中基于双深度 Q 学习的高能效资源分配方法[J]. 通信学报, 2020,41(12): 148-161.
	YU P , ZHANG J Y , LI W J ,et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on Communications, 2020,41(12): 148-161.
[22]	3GPP. Further advancements for E-UTRA physical layer aspects:TR36.814 (V9.0.0)[S]. 2010.
[23]	3GPP. Service requirements for 5G system (Release 15):TS 22.261 V18.3.0[S]. 2021.
[24]	SUN G L , XIONG K , BOATENG G O ,et al. Resource slicing and customization in RAN with dueling deep Q-network[J]. Journal of Network and Computer Applications, 2020,157:102573.

参数	eMBB	URLLC	VoLTE
带宽/MHz	40	40	40
轮询调度/ms	0.5	0.5	0.5
用户数量	400	600	200
用户移动速度/(m·s^-1)	均匀分布[1,5 ]	均匀分布[6,10]	均匀分布[0,1]
数据包请求到达间隔分布	截断帕累托分布	指数分布	均匀分布
	[指数系数=1.2, mean=6 ms, max=12.5 ms]	[mean=180 ms]	[min= 0 ms, max=160 ms]
数据包的大小分布	截断帕累托分布	0.3 MB	40 B
	[指数系数=1.2, mean=100 B, max=250 B
切片要求速率	100 Mbit/s	10 Mbit/s	51 kbit/s
时延/ms	10	3	10

面向B5G多业务场景基于D3QN的双时间尺度网络切片算法

Dual time scale network slicing algorithm based on D3QN for B5G multi-service scenarios

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 24

相关文章 15

Metrics

推荐阅读 0

[1]	王再见, 谷慧敏. 基于联合优化的网络切片资源分配策略[J]. 通信学报, 2023, 44(5): 234-245.
[2]	余雪勇, 邱礼翔, 宋家宁, 朱洪波. 无人机辅助边缘计算中安全通信与能效优化策略[J]. 通信学报, 2023, 44(3): 45-54.
[3]	李国军, 侯旭, 叶昌荣, 罗一平. 短波通信接入网广域协作资源分配算法[J]. 通信学报, 2023, 44(2): 112-121.
[4]	龙隆, 刘子辰, 陆在旺, 张玉成, 李蕾. 移动边缘网络下服务缓存与资源分配联合优化策略[J]. 通信学报, 2023, 44(1): 64-74.
[5]	朱晓荣, 陈康. 基于细粒度切片的6G网络弹性切换算法研究[J]. 通信学报, 2022, 43(9): 148-156.
[6]	范绍帅, 吴剑波, 田辉. 面向能量受限工业物联网设备的联邦学习资源管理[J]. 通信学报, 2022, 43(8): 65-77.
[7]	王莉, 魏青, 徐连明, 沈渊, 张平, 费爱国. 面向通信-导航-感知一体化的应急无人机网络低能耗部署研究[J]. 通信学报, 2022, 43(7): 1-20.
[8]	秦鹏, 和昊婷, 赵雄文, 伏阳, 张钰, 王淼, 王硕, 武雪. 基于停放车辆路边单元环境感知的车联网资源高效分配[J]. 通信学报, 2022, 43(7): 113-125.
[9]	廖建新, 付霄元, 戚琦, 王敬宇, 孙海峰. 6G-ADM：基于知识空间的6G网络管控体系[J]. 通信学报, 2022, 43(6): 3-15.
[10]	张达敏, 王义, 邹诚诚, 赵沛雯, 张琳娜. 认知异构蜂窝网络中改进蜉蝣算法的资源分配策略[J]. 通信学报, 2022, 43(6): 156-167.
[11]	亓伟敬, 宋清洋, 郭磊. 面向软件定义多模态车联网的双时间尺度RAN切片资源分配[J]. 通信学报, 2022, 43(4): 60-70.
[12]	朱政宇, 侯庚旺, 黄崇文, 孙钢灿, 郝万明, 梁静. 基于并行CNN的RIS辅助D2D保密通信系统资源分配算法[J]. 通信学报, 2022, 43(3): 172-179.
[13]	孙巍, 宋清洋, 郭磊. 智能反射表面辅助的无线携能通信网络资源分配算法[J]. 通信学报, 2022, 43(2): 34-43.
[14]	孙俨, 熊翱, 蒋承伶, 王威, 于东晓, 郭少勇. 基于区块链的计算与无线通信资源联合管理双向拍卖模型[J]. 通信学报, 2022, 43(11): 14-25.
[15]	苏新, 孟蕾蕾, 周一青, CELIMUGE Wu. 基于深度强化学习的海洋移动边缘计算卸载方法[J]. 通信学报, 2022, 43(10): 133-145.