基于DQN的车载边缘网络任务分发卸载算法

doi:10.11959/j.issn.1000-436x.2020160

通信学报 ›› 2020, Vol. 41 ›› Issue (10): 172-178.doi: 10.11959/j.issn.1000-436x.2020160

基于DQN的车载边缘网络任务分发卸载算法

赵海涛^1,^2,³,张唐伟^1,^2,³,陈跃^1,^2,³,赵厚麟^1,^2,³,朱洪波^1,^2,³

¹ 南京邮电大学教育部泛在网络健康服务系统工程研究中心，江苏南京 210003
² 南京邮电大学江苏省无线通信重点实验室，江苏南京 210003
³ 南京邮电大学通信与信息工程学院，江苏南京 210003

修回日期:2020-06-13 出版日期:2020-10-25 发布日期:2020-11-05
作者简介:赵海涛（1983– ），男，江苏南京人，博士，南京邮电大学教授、硕士生导师，主要研究方向为无线多媒体建模、容量预测和无线网络编码等|张唐伟（1994– ），男，安徽安庆人，南京邮电大学硕士生，主要研究方向为移动边缘计算、无线系统资源分配等|陈跃（1996– ），男，安徽宿州人，南京邮电大学硕士生，主要研究方向为物联网路由优化和边缘计算等|赵厚麟（1950– ），男，江苏高邮人，博士，南京邮电大学兼职教授、博士生导师，主要研究方向为IPv6技术标准实现及其在下一代信息网络中的应用、下一代网络（NGN）关键技术及标准化研究等|朱洪波（1956– ），男，江苏扬州人，博士，南京邮电大学教授、博士生导师，主要研究方向为无线通信与电磁兼容、移动通信、宽带无线技术等
基金资助:
国家自然科学基金资助项目(61771252);江苏省自然科学基金资助项目(BK20171444);江苏省高等学校自然科学研究基金资助项目(18KJA510005);江苏省科技成果转化专项基金资助项目(BA2019058);江苏省“333高层次人才培养工程”基金资助项目(JSCX17_0224)

Task distribution offloading algorithm of vehicle edge network based on DQN

Haitao ZHAO^1,^2,³,Tangwei ZHANG^1,^2,³,Yue CHEN^1,^2,³,Houlin ZHAO^1,^2,³,Hongbo ZHU^1,^2,³

¹ Ministry of Education Ubiquitous Network Health Service System Engineering Research Center,Nanjing 210003,China
² Jiangsu Key Wireless Communication Laboratory,Nanjing 210003,China
³ College of Communication and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003,China

Revised:2020-06-13 Online:2020-10-25 Published:2020-11-05
Supported by:
The National Natural Science Foundation of China(61771252);The Natural Science Foundation of Jiangsu Province(BK20171444);The Major Natural Science Research Project of Jiangsu Higher Education Institutions(18KJA510005);Jiangsu Province Special Fund Project for Transformation of Scientific and Technological Achievements(BA2019058);The Object of Jiangsu Province “333 High-level Talent Training Project”(JSCX17_0224)

摘要/Abstract

摘要：

为实现车辆终端用户任务执行时延与处理速率、能耗的最佳均衡关系，针对车联网的边缘接入环境，提出了一种基于深度 Q 网络（DQN）的计算任务分发卸载算法。首先根据层次分析法对不同车辆终端的计算任务进行优先级划分，从而为计算任务处理速率赋予不同的权重建立关系模型；然后引入基于深度Q网络的边缘计算方法，以计算任务处理速率加权和为优化目标建立任务卸载模型；最后建立基于 DQN 的车辆终端自主最优任务卸载策略，最大化卸载决策制定模型的长期效用。仿真结果表明，相比Q学习算法，所提算法有效提高了任务执行效率。

关键词: 车联网, 移动边缘计算, 计算卸载, 深度Q网络, 计算速率

Abstract:

In order to achieve the best balance between latency,computational rate and energy consumption,for a edge access network of IoV,a distribution offloading algorithm based on deep Q network (DQN) was considered.Firstly,these tasks of different vehicles were prioritized according to the analytic hierarchy process (AHP),so as to give different weights to the task processing rate to establish a relationship model.Secondly,by introducing edge computing based on DQN,the task offloading model was established by making weighted sum of task processing rate as optimization goal,which realized the long-term utility of strategies for offloading decisions.The performance evaluation results show that,compared with the Q-learning algorithm,the average task processing delay of the proposed method can effectively improve the task offload efficiency.

Key words: IoV, MEC, computational offloading, DQN, computational rate

中图分类号:

TP399

赵海涛,张唐伟,陈跃,赵厚麟,朱洪波. 基于DQN的车载边缘网络任务分发卸载算法[J]. 通信学报, 2020, 41(10): 172-178.

Haitao ZHAO,Tangwei ZHANG,Yue CHEN,Houlin ZHAO,Hongbo ZHU. Task distribution offloading algorithm of vehicle edge network based on DQN[J]. Journal on Communications, 2020, 41(10): 172-178.

图/表 5

参考文献 20

[1]	NING Z , FENG Y F , COLLOTTA M ,et al. Deep learning in edge of vehicles:exploring trirelationship for data transmission[J]. IEEE Transactions on Industrial Informatics, 2019,15(10): 5737-5746.
[2]	ZHOU H , XU W , CHEN J . Evolutionary V2X technologies toward the Internet of vehicles:challenges and opportunities[J]. Proceedings of the IEEE, 2020,108(2): 308-323.
[3]	POPLI S , JHA R K , JAIN S . A survey on energy efficient narrowband Internet of things (NBloT):architecture,application and challenges[J]. IEEE Access, 2019(7): 16739-16776.
[4]	谢人超, 廉晓飞, 贾庆民 ,等. 移动边缘计算卸载技术综述[J]. 通信学报, 2018,39(11): 138-155.
	XIE R C , LIAN X F , JIA Q M ,et al. Overview of mobile edge computing offload technology[J]. Journal on Communications, 2018,39(11): 138-155.
[5]	XIE J , JIA Y , CHEN Z ,et al. Efficient task completion for parallel offloading in vehicular fog computing[J]. China Communications, 2019,16(11): 42-45.
[6]	HUANG K B , ZHU G X , YOU C S ,et al. Communication,computing,and learning on the edge[C]// 2018 IEEE International Conference on Communication Systems. Piscataway:IEEE Press, 2018: 268-273.
[7]	MAO Y , YOU C , ZHANG J ,et al. A survey on mobile edge computing:the communication perspective[J]. IEEE Communications Surveys ＆ Tutorials, 2017,19(4): 2322-2358.
[8]	ZHANG J , LETAIEF K B . Mobile edge intelligence and computing for the Internet of vehicles[J]. Proceedings of the IEEE, 2020,108(2): 246-261.
[9]	KETYKO I , KECSKES L , NEMES C ,et al. Multi-user computation offloading as multiple knapsack problem for 5G mobile edge computing[C]// 2016 European Conference on Networks and Communications. Piscataway:IEEE Press, 2016: 225-229.
[10]	ZAHO J , LI Q , GONG Y ,et al. Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks[J]. IEEE Transactions on Vehicular Technology, 2019,68(8): 7944-7956.
[11]	MENG X , WANG W , WANG Y ,et al. Closed-form delay-optimal computation offloading[J]. Mobile Edge Computing Systems, 2019,18(10): 4653-4667.
[12]	DU J , ZHAO L , FENG J ,et al. Computation offloading and resource allocation in mixed fog/cloud computing systems with min-max fairness guarantee[J]. IEEE Transactions on Communications, 2017,66(4): 1-3.
[13]	郭辉, 芮兰兰, 高志鹏 . 车辆终端边缘网络中基于多参数 MDP 模型的动态服务迁移策略[J]. 通信学报, 2020,41(1): 1-14.
	GUO H , RUI L L , GAO Z P . Dynamic service migration strategy based on multi-parameter MDP model in vehicle edge network[J]. Journal on Communications, 2020,41(1): 1-14.
[14]	LI J , GAO H , LYU T ,et al. Deep reinforcement learning based computation offloading and resource allocation for MEC[C]// 2018 IEEE Wireless Communications and Networking Conference. Piscataway:IEEE Press, 2018: 1-6.
[15]	ARULKUMARAN K , DEISENROTH M P , BRUNDAGE M ,et al. Deep reinforcement learning:a brief survey[J]. IEEE Signal Processing Magazine, 2017,34(6): 26-38.
[16]	MAO Q , FEI H , QI H . Deep learning for intelligent wireless networks:a comprehensive survey[J]. IEEE Communications Surveys ＆ Tutorials, 2018,20(4): 2595-2621.
[17]	HUANG Y L , SUN W L . An AHP-based risk assessment for an industrial IoT cloud[C]// 2018 IEEE International Conference on Software Quality,Reliability and Security Companion. Piscataway:IEEE Press, 2018: 637-638.
[18]	余翔, 刘一勋, 石雪琴 ,等. 车联网场景下的移动边缘计算卸载策略[J]. 计算机工程, 2020,doi:10.19678/j.issn.1000-3428.0056850.
	YU X , LIU Y X , SHI X Q ,et al. Offloading strategy for mobile edge computing in Vehicle Network[J]. Computer Engineering, 2020,doi:10.19678/j.issn.1000-3428.0056850.
[19]	陈山枝, 胡金玲, 时岩 . LTE-V2X车联网技术、标准与应用[J]. 电信科学, 2018,34(4): 1-11.
	CHEN S Z , HU J L , SHI Y . LTE-V2X car networking technology,standards and applications[J]. Telecommunications Science, 2018,34(4): 1-11.
[20]	王寒松 . 车联网中基于 MEC 的计算任务卸载策略研究[D]. 北京:北京邮电大学, 2019.
	WANG H S . Research of computing offloading scheme for MEC-enabled vehicular networks[D]. Beijing:Beijing University of Posts and Telecommunications, 2019.

基于DQN的车载边缘网络任务分发卸载算法

Task distribution offloading algorithm of vehicle edge network based on DQN

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 20

相关文章 15

Metrics

推荐阅读 0

[1]	张海波, 兰凯, 陈舟, 王汝言, 邹灿, 王明月. 车联网中基于环的匿名高效批量认证与组密钥协商协议[J]. 通信学报, 2023, 44(6): 103-116.
[2]	张海波, 曹钰坤, 刘开健, 王汝言. 车联网中基于区块链的分布式信任管理方案[J]. 通信学报, 2023, 44(5): 148-157.
[3]	刘雪娇, 钟强, 夏莹杰. 基于双层分片区块链的车联网跨信任域高效认证方案[J]. 通信学报, 2023, 44(5): 213-223.
[4]	周大成, 陈鸿昶, 何威振, 程国振, 扈红超. 基于深度强化学习的微服务多维动态防御策略研究[J]. 通信学报, 2023, 44(4): 50-63.
[5]	苏新, 张桂福, 行鸿彦, Zenghui Wang. 基于平衡生成对抗网络的海洋气象传感网入侵检测研究[J]. 通信学报, 2023, 44(4): 124-136.
[6]	余雪勇, 邱礼翔, 宋家宁, 朱洪波. 无人机辅助边缘计算中安全通信与能效优化策略[J]. 通信学报, 2023, 44(3): 45-54.
[7]	刘雪娇, 曹天聪, 夏莹杰. 区块链架构下高效的车联网跨域数据安全共享研究[J]. 通信学报, 2023, 44(3): 186-197.
[8]	张雷, 王玉, 田建杰, 张琳, 章天骄. 基于IRS辅助的MIMO车联网系统联合波束成形设计[J]. 通信学报, 2023, 44(2): 59-69.
[9]	范伟, 彭诚, 朱大立, 王雨晴. 移动边缘计算网络下基于静态贝叶斯博弈的入侵响应策略研究[J]. 通信学报, 2023, 44(2): 70-81.
[10]	赵庶旭, 韦萍, 王小龙. 多任务并发边缘计算环境中最优联盟结构生成策略[J]. 通信学报, 2023, 44(2): 172-184.
[11]	夏景明, 刘玉风, 谈玲. 基于蜂窝网络的多无人机能量消耗最优化算法研究[J]. 通信学报, 2023, 44(2): 185-197.
[12]	龙隆, 刘子辰, 陆在旺, 张玉成, 李蕾. 移动边缘网络下服务缓存与资源分配联合优化策略[J]. 通信学报, 2023, 44(1): 64-74.
[13]	曾嵘, 杭潇. 车联网环境下可重构智能反射面辅助无线信道估计算法[J]. 通信学报, 2022, 43(8): 142-150.
[14]	程翔, 张浩天, 杨宗辉, 黄子蔚, 李思江, 余安澜. 车联网通信感知一体化研究：现状与发展趋势[J]. 通信学报, 2022, 43(8): 188-202.
[15]	秦鹏, 和昊婷, 赵雄文, 伏阳, 张钰, 王淼, 王硕, 武雪. 基于停放车辆路边单元环境感知的车联网资源高效分配[J]. 通信学报, 2022, 43(7): 113-125.