通信学报 ›› 2021, Vol. 42 ›› Issue (5): 191-204.doi: 10.11959/j.issn.1000-436x.2021071

• 学术通信 • 上一篇    下一篇

基于深度强化学习的能源互联网智能巡检任务分配机制

徐思雅1, 邢逸斐1, 郭少勇1, 杨超2, 邱雪松1, 孟洛明1   

  1. 1 北京邮电大学网络技术与交换重点实验室,北京 100876
    2 国网辽宁省电力有限公司信息通信分公司,辽宁 沈阳 110004
  • 修回日期:2021-03-10 出版日期:2021-05-25 发布日期:2021-05-01
  • 作者简介:徐思雅(1988- ),女,北京人,博士,北京邮电大学讲师,主要研究方向为信息通信网络管理、SDN/NFV、移动边缘计算和人工智能等
    邢逸斐(1997- ),男,河北保定人,北京邮电大学硕士生,主要研究方向为智能电网网络管理、移动边缘计算等
    郭少勇(1985- ),男,河北邢台人,博士,北京邮电大学副教授,主要研究方向为区块链、物联网等
    杨超(1988- ),男,山东平度人,博士,国网辽宁省电力有限公司工程师,主要研究方向为电力物联网、人工智能工程化应用等
    邱雪松(1973- ),男,江西上饶人,博士,北京邮电大学教授、博士生导师,主要研究方向为网络与业务管理、物联网与区块链
    孟洛明(1955- ),男,河南洛阳人,博士,北京邮电大学教授、博士生导师,主要研究方向为通信网、网络管理、通信软件等
  • 基金资助:
    国家重点研发计划基金资助项目(2019YFB2102302);国家自然科学基金资助项目(61702048);工业互联网创新发展工程基金资助项目(基于泛在电力物联网的工业互联网测试床)

Deep reinforcement learning based task allocation mechanism for intelligent inspection in energy Internet

Siya XU1, Yifei XING1, Shaoyong GUO1, Chao YANG2, Xuesong QIU1, Luoming MENG1   

  1. 1 State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
    2 Information and Communication Branch, State Grid Liaoning Electric Power Co., Ltd., Shenyang 110004, China
  • Revised:2021-03-10 Online:2021-05-25 Published:2021-05-01
  • Supported by:
    The National Key Research and Development Program of China(2019YFB2102302);The National Natural Science Foundation of China(61702048);Innovative Development Project of Industrial Internet (Construction of Industrial In-ternet Test Bed Based on Ubiquitous Power Internet of Things)

摘要:

在能源互联网中引入无人机进行电力线路巡查,并借助移动边缘计算技术实现巡检任务的接入和处理,可降低服务成本,提高工作效率。但是,由于无人机数据传输需求和地理位置的动态变化,易造成边缘服务器负载不均衡,致使巡检业务处理时延和网络能耗较高。为解决以上问题,提出基于深度强化学习的能源互联网智能巡检任务分配机制。首先,综合考虑无人机和边缘节点的运动轨迹、业务差异化的服务需求、边缘节点有限的服务能力等,建立面向时延、能耗等多目标联合优化的双层边缘网络任务卸载模型。进而,基于 Lyapunov 优化理论和双时间尺度机制,采用近端策略优化的深度强化学习算法,对固定边缘汇聚层和移动边缘接入层边缘节点间的连接关系和卸载策略进行求解。仿真结果表明,所提机制能够在保证系统稳定的情况下降低服务时延和系统能耗。

关键词: 巡检无人机, 任务卸载, 近端策略优化, 李雅普诺夫优化, 人工智能

Abstract:

In order to reduce the cost and improve efficiency of power line inspection, UAV (unmanned aerial vehicle), which use mobile edge computing technology to access and process service data, are used to inspect power lines in the energy internet.However, due to the dynamic changes of UAV data transmission demand and geographical location, the edge server load will be unbalanced, which causes higher service processing delay and network energy consumption.Thus, an intelligent inspection task allocation mechanism for energy internet based on deep reinforcement learning was proposed.First, a two-layer edge network task offloading model was established to archive joint optimization of multi-objectives, such as delay and energy consumption.It was designed by comprehensively considering the route of UAV and edge nodes, different demands of services and limited service capabilities of edge nodes.Furthermore, based on Lyapunov optimization theory and dual-time-scaled mechanism, proximal policy optimization algorithm based deep reinforcement learning was used to solve the connection relationship and offloading strategy of edge servers between fixed edge sink layer and mobile edge access layer.The simulation results show that, the proposed mechanism can reduce the service request delay and system energy consumption while ensuring the stability of system.

Key words: patrol UAV, ask offloading, proximal policy optimization, Lyapunov optimization, artificial intelligence

中图分类号: 

No Suggested Reading articles found!