通信学报 ›› 2021, Vol. 42 ›› Issue (6): 118-130.doi: 10.11959/j.issn.1000-436x.2021104

• 学术论文 • 上一篇    

基于多智能体元强化学习的车联网协同服务缓存和计算卸载

宁兆龙1,2, 张凯源2, 王小洁1, 郭磊1   

  1. 1 重庆邮电大学通信与信息工程学院,重庆 400065
    2 大连理工大学软件学院,辽宁 大连 116620
  • 修回日期:2021-03-24 出版日期:2021-06-01 发布日期:2021-06-01
  • 作者简介:宁兆龙(1986− ),男,辽宁沈阳人,博士,重庆邮电大学教授,主要研究方向为边缘计算、车联网、网络优化等
    张凯源(1994− ),男,黑龙江哈尔滨人,大连理工大学硕士生,主要研究方向为人工智能、边缘计算
    王小洁(1988− ),女,河北张家口人,博士,重庆邮电大学特聘教授,主要研究方向为物联网、人工智能、边缘计算
    郭磊(1980− ),男,四川眉山人,博士,重庆邮电大学教授,主要研究方向为网络优化、网络通信、光网络等
  • 基金资助:
    国家重点研发计划基金资助项目(2018YFE0206800);国家自然科学基金资助项目(62025105);国家自然科学基金资助项目(61971084);国家自然科学基金资助项目(62001073);重庆英才计划基金资助项目(CQYC2020058659)

Cooperative service caching and peer offloading in Internet of vehicles based on multi-agent meta-reinforcement learning

Zhaolong NING1,2, Kaiyuan ZHANG2, Xiaojie WANG1, Lei GUO1   

  1. 1 School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
    2 School of Software, Dalian University of Technology, Dalian 116620, China
  • Revised:2021-03-24 Online:2021-06-01 Published:2021-06-01
  • Supported by:
    The National Key Research and Development Program of China(2018YFE0206800);The National Natural Science Foundation of China(62025105);The National Natural Science Foundation of China(61971084);The National Natural Science Foundation of China(62001073);Chongqing Talent Program(CQYC2020058659)

摘要:

为了降低求解优化问题的难度,提出一种双层的多路侧单元(RSU)协同缓存框架将问题进行解耦。外层采用多智能体元强化学习方法,在每个本地智能体进行决策学习的同时,采用长短期记忆网络作为元智能体来平衡本地决策并加速学习过程,从而得到最优的 RSU 缓存策略;内层采用拉格朗日乘子法求解最佳协同卸载策略,实现 RSU 间的任务分配。基于杭州真实交通数据的实验表明,所提方法具有理想的能效性能,并且能够在大规模任务流下保持网络稳健性。

关键词: 车联网, 边缘服务缓存, 协同卸载, 元强化学习

Abstract:

In order to reduce computation complexity, a two-layer mutli-RSU (road side unit) service caching and peer offloading algorithm (MPO) was proposed to decouple the optimization problem.In the designed MPO, the outer layer utilized multi-agent meta-reinforcement learning, which established long short-term memory (LSTM) network as the meta-agent to balance decisions of local agents and accelerate learning progress.The inner layer utilized lagrange multiplier method to achieve optimal decision for RSU peer offloading.Experimental results based on real traffic data in Hangzhou demonstrate that the proposed method outperforms other methods and remains robust under large-scale workloads.

Key words: Internet of vehicles, edge service caching, cooperative offloading, meta-reinforcement learning

中图分类号: 

No Suggested Reading articles found!