基于深度Q学习的蜂窝车联网边路资源分配算法

doi:10.11959/j.issn.2096-6652.202309

智能科学与技术学报 ›› 2023, Vol. 5 ›› Issue (1): 83-91.doi: 10.11959/j.issn.2096-6652.202309

基于深度Q学习的蜂窝车联网边路资源分配算法

许辉

中兴通讯股份有限公司终端事业部，四川成都 610041

修回日期:2023-02-24 出版日期:2023-03-15 发布日期:2023-03-01
作者简介:许辉（1972- ），男，博士，中兴通讯股份有限公司终端事业部资深标准研究员，主要研究方向为5G/6G中的物联网、车联网和广播多播服务等标准研究
基金资助:
国家重点研发计划资助项目(2018YFB1308600)

Sidelink resource allocation algorithm of C-V2X based on deep Q learning

Hui XU

Terminal Business Division, ZTE Corporation, Chengdu 610041, China

Revised:2023-02-24 Online:2023-03-15 Published:2023-03-01
Supported by:
The National Key Research and Development Program of China(2018YFB1308600)

摘要/Abstract

摘要：

针对蜂窝车联网系统中不同优先级业务的边路资源自主选择分配，分析了基于参考信号能量的自主选择算法流程，设计了能量门限方程；针对能量方程参数估计问题，将基于能量的自主选择算法与深度Q学习算法结合，通过有限次算法迭代得到能量门限方程的最优参数值。仿真结果表明，基于深度Q学习的边路资源分配算法可以满足不同优先级车联网业务的边路资源需求，同时提高系统分组接收率。

关键词: 蜂窝车联网, 边路, 自主资源选择, 机器学习, 分组接收率

Abstract:

For the sidelink resource autonomous selection scheme of different priority services in celluar-vehicle to everything (C-V2X) system, the procedure of autonomous selection algorithm based on reference signal energy was analyzed, and the energy threshold equation was designed.For the energy equation parameter estimation problem, energy-based autonomous selection algorithm was combined with deep Q learning algorithm, and the optimal parameter value of the energy threshold equation was obtained by iteration of the finite-degree algorithm.Simulation results showed that the side resource allocation algorithm based on deep Q learning could ensure the side resource requirements of V2X services with different priorities, and improve the packet reception ratio performance of the system.

Key words: C-V2X, sidelink, autonomous resource selection, machine learning, PRR

中图分类号:

TN929.5

许辉. 基于深度Q学习的蜂窝车联网边路资源分配算法[J]. 智能科学与技术学报, 2023, 5(1): 83-91.

Hui XU. Sidelink resource allocation algorithm of C-V2X based on deep Q learning[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 83-91.

图/表 11

图1

图2

图3

图4

图5

表1

图6

图7

图8

图9

图10

参考文献 19

[1]	陈山枝, 时岩, 胡金玲 . 蜂窝车联网(C-V2X)综述[J]. 中国科学基金, 2020,34(2): 179-185.
	CHEN S Z , SHI Y , HU J L . Cellular vehicle to everything (C-V2X):a review[J]. Bulletin of National Natural Science Foundation of China, 2020,34(2): 179-185.
[2]	葛雨明 . 我国LTE-V2X标准化及测试验证进展[J]. 移动通信, 2019,43(11): 36-39.
	GE Y M . Progress on LTE-V2X standardization and testing in China[J]. Mobile Communications, 2019,43(11): 36-39.
[3]	杨波, 张莹, 蔡之骏 ,等. 基于LTE-V的车载V2X系统研究[J]. 移动通信, 2019,43(11): 75-80.
	YANG B , ZHANG Y , CAI Z J ,et al. Research on the vehicle V2X system based on LTE-V[J]. Mobile Communications, 2019,43(11): 75-80.
[4]	刘继军, 邹山花, 卢先领 . MEC 中资源分配与卸载决策联合优化策略[J]. 计算机科学与探索, 2021,15(5): 848-858.
	LIU J J , ZOU S H , LU X L . Joint optimization strategy of resource allocation and unloading decision in MEC[J]. Computer Science and Exploration, 2021,15(5): 848-858.
[5]	余翔, 陈晓东, 王政 ,等. 基于 LTE-V2X 的联网资源分配算法[J]. 计算机工程, 2021,47(2): 188-192.
	YU X , CHEN X D , WANG Z ,et al. Network resource allocation algorithm based on LTE-V2X[J]. Computer Engineering, 2021,47(2): 188-192.
[6]	NGUYEN L H , NGUYEN V L , KUO J J . Efficient reinforcement learning-based transmission control for mitigating channel congestion in 5G V2X sidelink[J]. IEEE Access, 2022,10: 62268-62281.
[7]	陆岩 . C-V2X 通信分布式无线资源调度关键技术研究[D]. 上海:东华大学, 2022.
	LU Y . Research on key technologies of distributed radio resource scheduling in C-V2X communication[D]. Shanghai:Donghua University, 2022.
[8]	杨天, 杨军 . 移动边缘计算中的卸载决策与资源分配策略[J]. 计算机工程, 2021,47(2): 19-25.
	YANG T , YANG J . Unloading decision and resource allocation strategy in moving edge computing[J]. Computer Engineering, 2021,47(2): 19-25.
[9]	HAMZAH H , LE D T , KIM M ,et al. Location-aware task offloading for MEC-based high mobility service[C]// Proceedings of 2021 International Conference on Information Networking. Piscataway:IEEE Press, 2021: 708-711.
[10]	CHEN H M , WANG S F , WANG P ,et al. Deep Q-learning for intelligent band coordination in 5G heterogeneous network supporting V2X communication[J]. Wireless Communications and Mobile Computing, 2022: 1-12.
[11]	张兴龙, 陆阳, 李文璋 ,等. 基于滚动时域强化学习的智能车辆侧向控制算法[J]. 自动化学报, 2022,45: 1-12.
	ZHANG X L , LU Y , LI W Z ,et al. Lateral control algorithm for intelligent vehicles based on rolling time domain reinforcement learning[J]. Acta Automatica Sinica, 2022,45: 1-12.
[12]	裴晓飞, 莫烁杰, 陈祯福 ,等. 基于TD3算法的人机混驾交通环境自动驾驶汽车换道研究[J]. 中国公路学报, 2021,34(11): 246-254.
	PEI X F , MO S J , CHEN Z F ,et al. Lane changing of autonomous vehicle based on TD3 algorithm in human-machine hybrid driving environment[J]. China Journal of Highway and Transport, 2021,34(11): 246-254.
[13]	田康, 于镝, 李擎 ,等. 基于改进TD3的自动驾驶车道保持决策方法[J]. 北京交通大学学报, 2022,46(5): 84-94.
	TIAN K , YU D , LI Q ,et al. Lane keeping decision-making method of autonomous vehicles based on improved TD3[J]. Journal of Beijing Jiaotong University, 2022,46(5): 84-94.
[14]	罗薇, 汪梦珍, 许玲 . 车联网高层协议关键技术[J]. 中兴通讯技术, 2020,26(1): 35-39.
	LUO W , WANG M Z , XU L . Main technologies adopted in high layer protocol for Internet of vehicles[J]. ZTE Technology Journal, 2020,26(1): 35-39.
[15]	SANSANUM H . Technology and device for autonomous resource selection for internet of vehicles (V2X) transmission[P]. 2020.
[16]	3GPP. Evolved universal terrestrial radio access (E-UTRA) and evolved universal terrestrial radio access network (E-UTRAN); overall description:TS 36.300[S]. 2018: 289-302.
[17]	3GPP. Evolved universal terrestrial radio access (E-UTRA); physical layer procedures:TS 36.213[S]. 2018: 422-447.
[18]	刘伯阳, 马杰, 白静 ,等. 基于MDP的协作认知边缘计算网络资源分配方案[J]. 西安邮电大学学报, 2022,27(1): 23-28.
	LIU B Y , MA J , BAI J ,et al. MDP based resource allocation scheme for cooperative cognitive mobile edge computing networks[J]. Journal of Xi’an University of Posts and Telecommunications, 2022,27(1): 23-28.
[19]	3GPP. Study on LTE-based V2X services:TR 36.885[S]. 2016: 60-72.

参数	优先级p1,p2	动作集合a_t	b	折扣γ	学习因子α	资源分配信息SA	子信道	数据包
值	{1,2,3,4,5,6,7,8}	{0,1,2,3,…,10}	-87	0.5	0.9	30 bit（发两次）	10 PRB	190/300 B

基于深度Q学习的蜂窝车联网边路资源分配算法

Sidelink resource allocation algorithm of C-V2X based on deep Q learning

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 19

相关文章 4

Metrics

推荐阅读 0

[1]	胡东伟, 冯晓璐. 大脑建模的理论框架及热点问题[J]. 智能科学与技术学报, 2021, 3(4): 412-434.
[2]	刘晨熙, 刘炳宏, 张贤, 龙新南, 彭木根. 面向智能服务的雾无线接入网络：原理、技术与挑战[J]. 智能科学与技术学报, 2021, 3(1): 10-17.
[3]	胡毅, 瞿博阳, 梁静, 王杰, 王艳丽. 进化集成学习算法综述[J]. 智能科学与技术学报, 2021, 3(1): 18-35.
[4]	曹汉平,张晓晶,祝睿杰,黄潇拉. 数字金融时代机器学习模型在实时反欺诈中的应用与实践[J]. 智能科学与技术学报, 2019, 1(4): 342-351.