基于深度强化学习的软件定义网络QoS优化

doi:10.11959/j.issn.1000-436x.2019227

Abstract

Abstract:

To solve the problem that the QoS optimization schemes which based on heuristic algorithm degraded often due to the mismatch between parameters and network characteristics in software-defined networking scenarios,a software-defined networking QoS optimization algorithm based on deep reinforcement learning was proposed.Firstly,the network resources and state information were integrated into the network model,and then the flow perception capability was improved by the long short-term memory,and finally the dynamic flow scheduling strategy,which satisfied the specific QoS objectives,were generated in combination with deep reinforcement learning.The experimental results show that,compared with the existing algorithms,the proposed algorithm not only ensures the end-to-end delay and packet loss rate,but also improves the network load balancing by 22.7% and increases the throughput by 8.2%.

Key words: software-defined networking, deep reinforcement learning, long short-term memory, quality of service

CLC Number:

TP393

Julong LAN,Xueshuai ZHANG,Yuxiang HU,Penghao SUN. Software-defined networking QoS optimization based on deep reinforcement learning[J]. Journal on Communications, 2019, 40(12): 60-67.

Figures/Tables 7

References 20

[1]	MCKEOWN N . Software-defined networking[J]. INFOCOM Keynote Talk, 2009,17(2): 30-32.
[2]	张朝昆, 崔勇, 唐翯祎 ,等. 软件定义网络（SDN）研究进展[J]. 软件学报, 2015,26(1): 62-81.
	ZHANG C K , CUI Y , TANG H Y ,et al. State-of-the-art survey on software-defined-networking (SDN)[J]. Journal of Software, 2015,26(1): 62-81.
[3]	董谦, 李俊, 马宇翔 ,等. 软件定义网络中基于分段路由的流量调度方法[J]. 通信学报, 2018,39(11): 23-35.
	DONG Q , LI J , MA Y X ,et al. Traffic scheduling method based on segment routing in software-defined networking[J]. Journal on Com-munications, 2018,39(11): 23-35.
[4]	HARTMAN T , HASSIDIM A , KAPLAN H ,et al. How to split a flow?[C]// 2012 Proceedings IEEE INFOCOM. IEEE, 2012: 828-836.
[5]	ZHANG S R , . Valiant load-balancing:building networks that can support all traffic matrices[M]// London:Algorithms for Next Generation Networks. Springer, 2010: 19-30.
[6]	SUTTON R S , BARTO A G . Reinforcement learning:an introduction[M]. MIT Press, 2018.
[7]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553):436.
[8]	MAO H , NETRAVALI R , ALIZADEH M . Neural adaptive video streaming with pensieve[C]// The Conference of the ACM Special Interest Group on Data Communication. ACM, 2017: 197-210.
[9]	XIAO S , HE D , GONG Z . Deep-q:traffic-driven QoS inference using deep generative network[C]// The 2018 Workshop on Network Meets AI ＆ ML. ACM, 2018: 67-73.
[10]	CHEN L , CHEN K , BAI W ,et al. Scheduling mix-flows in commodity datacenters with karuna[C]// The 2016 ACM SIGCOMM Conference. ACM, 2016: 174-187.
[11]	ONGARO F , CERQUEIRA E , FOSCHINI L ,et al. Enhancing the quality level support for real-time multimedia applications in software-defined networks[C]// 2015 International Conference on Computing,Networking and Communications (ICNC). IEEE, 2015: 505-509.
[12]	ALIZADEH M , EDSALL T , DHARMAPURIKAR S ,et al. CONGA:distributed congestion-aware load balancing for datacenters[J]. ACM SIGCOMM Computer Communication Review, 2014,44(4): 503-514.
[13]	XU Z , TANG J , MENG J ,et al. Experience-driven networking:a deep reinforcement learning based approach[C]// IEEE INFOCOM 2018 IEEE Conference on Computer Communications. IEEE, 2018: 1871-1879.
[14]	XU Z , TANG J , YIN C ,et al. Experience-driven congestion control:when multi-path TCP meets deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2019,37(6): 1325-1336.
[15]	MCKEOWN N , ANDERSON T , BALAKRISHNAN H ,et al. OpenFlow:enabling innovation in campus networks[J]. ACM SIGCOMM Computer Communication Review, 2008,38(2): 69-74.
[16]	BOSSHART P , DALY D , GIBB G ,et al. P4:programming protocol-independent packet processors[J]. ACM SIGCOMM Computer Communication Review, 2014,44(3): 87-95.
[17]	ZHOU W , LI L , LUO M ,et al. REST API design patterns for SDN northbound API[C]// 2014 28th International Conference on Advanced Information Networking and Applications Workshops. IEEE, 2014: 358-365.
[18]	CLEMM A , CHANDRAMOULI M , KRISHNAMURTHY S . DNA:an SDN framework for distributed network analytics[C]// 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM). IEEE, 2015: 9-17.
[19]	LILLICRAP T P , HUNT J J , PRITZEL A ,et al. Continuous control with deep reinforcement learning[C]// Eighth International Conference on Learning Representations. ICLR, 2016: 187-200.
[20]	R?CKE H , . Optimal hierarchical decompositions for congestion minimization in networks[C]// The Fortieth Annual ACM Symposium on Theory of Computing. ACM, 2008: 255-264.

Metrics

Recommended 0

No Suggested Reading articles found!

属性	配置
操作系统	Ubuntu 16.04
CPU	Intel^?Xeon E5
GPU	NVIDIA Tesla P100
内存	64 GB

Software-defined networking QoS optimization based on deep reinforcement learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 20

Related Articles 15

Metrics

Recommended 0

[1]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[2]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[3]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[4]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[5]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[6]	Jianxun LIU, Linghang DING, Guosheng KANG, Buqing CAO, Yong XIAO. Joint QoS prediction for Web services based on deep fusion of features [J]. Journal on Communications, 2022, 43(7): 215-226.
[7]	Damin ZHANG, Yi WANG, Chengcheng ZOU, Peiwen ZHAO, Linna ZHANG. Resource allocation strategies for improved mayfly algorithm in cognitive heterogeneous cellular network [J]. Journal on Communications, 2022, 43(6): 156-167.
[8]	Ping WU, Chaowen CHANG, Zhibin ZUO, Yingying MA. Address overloading-based packet forwarding verification in SDN [J]. Journal on Communications, 2022, 43(3): 88-100.
[9]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[10]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[11]	Li’na DU, Li ZHUO, Shuo YANG, Jiafeng LI, Jing ZHANG. Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services [J]. Journal on Communications, 2021, 42(9): 205-217.
[12]	Ping WU, Chaowen CHANG, Yingying MA. Port address overloading based packet forwarding verification in SDN [J]. Journal on Communications, 2021, 42(7): 70-83.
[13]	Jian SHU, Qining WANG, Linlan LIU. UAV ad hoc network link prediction based on deep graph embedding [J]. Journal on Communications, 2021, 42(7): 137-149.
[14]	Wenchen HE, Shaoyong GUO, Xuesong QIU, Liandong CHEN, Suxiang ZHANG. Node selection method in federated learning based on deep reinforcement learning [J]. Journal on Communications, 2021, 42(6): 62-71.
[15]	Guolin SUN,Ruijie OU,Guisong LIU. Deep reinforcement learning-based resource reservation algorithm for emergency Internet-of-things slice [J]. Journal on Communications, 2020, 41(9): 8-20.