基于深度强化学习的转发效能感知流量调度算法

doi:10.11959/j.issn.1000-436x.2022148

Abstract

Abstract:

The software defined network separates the control plane from the data plane to achieve flexible traffic scheduling, which can use network resources more efficiently.However, with the increase of the number of flow entries, load rate, the number of connected hosts, and other factors, the forwarding efficiency of the SDN switch will be reduced, which will affect the end-to-end transmission delay.To solve the above problems, the forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning was proposed.First, the switch state was integrated into the perception model, and the mapping relationship between switch state information and forwarding efficiency was established based on neural network.Then, combined with network state and traffic information, traffic scheduling policy was generated through deep reinforcement learning.Finally, the expert samples generated by the shortest path and load balance algorithms could guide the model training, which enabled the model to learn knowledge from expert samples to improve performance and accelerated the training process.The experimental results show that the proposed algorithm not only reduces the average end-to-end transmission delay by 15.31%, but also ensures the overall load balance of the network, compared with other algorithms.

Key words: software defined network, deep reinforcement learning, traffic scheduling, forwarding efficiency aware, load balance

CLC Number:

TN911.5

Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning[J]. Journal on Communications, 2022, 43(8): 30-40.

Figures/Tables 6

References 27

[1]	周桐庆, 蔡志平, 夏竟 ,等. 基于软件定义网络的流量工程[J]. 软件学报, 2016,27(2): 394-417.
	ZHOU T Q , CAI Z P , XIA J ,et al. Traffic engineering for software defined networks[J]. Journal of Software, 2016,27(2): 394-417.
[2]	MENDIOLA A , ASTORGA J , JACOB E ,et al. A survey on the contributions of software-defined networking to traffic engineering[J]. IEEE Communications Surveys ＆ Tutorials, 2017,19(2): 918-953.
[3]	杨思锦, 庄雷, 宋玉 ,等. 多模态网络中时间敏感网络模态的智能调度机制[J]. 通信学报, 2022,43(5): 82-91.
	YANG S J , ZHUANG L , SONG Y ,et al. Intelligent scheduling mechanism of time-sensitive network modal in polymorphic network[J]. Journal on Communications, 2022,43(5): 82-91.
[4]	SARASWAT S , AGARWAL V , GUPTA H P ,et al. Challenges and solutions in software defined networking:a survey[J]. Journal of Network and Computer Applications, 2019,141: 23-58.
[5]	黄韬, 刘江, 汪硕 ,等. 未来网络技术与发展趋势综述[J]. 通信学报, 2021,42(1): 130-150.
	HUANG T , LIU J , WANG S ,et al. Survey of the future network technology and trend[J]. Journal on Communications, 2021,42(1): 130-150.
[6]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553): 436-444.
[7]	SUTTON R S , BARTO A G . Reinforcement learning:an introduction[M]. Massachusetts: MIT Press, 1998.
[8]	XIAO Y , LIU J , WU J W ,et al. Leveraging deep reinforcement learning for traffic engineering:a survey[J]. IEEE Communications Surveys ＆ Tutorials, 2021,23(4): 2064-2097.
[9]	XIE J F , YU F R , HUANG T ,et al. A survey of machine learning techniques applied to software defined networking (SDN):research issues and challenges[J]. IEEE Communications Surveys ＆ Tutorials, 2019,21(1): 393-430.
[10]	HARTMAN T , HASSIDIM A , KAPLAN H ,et al. How to split a flow?[C]// Proceedings of IEEE INFOCOM. Piscataway:IEEE Press, 2012: 828-836.
[11]	ZHANG J J , YE M H , GUO Z H ,et al. CFR-RL:traffic engineering with reinforcement learning in SDN[J]. IEEE Journal on Selected Areas in Communications, 2020,38(10): 2249-2259.
[12]	ZHANG J J , GUO Z H , YE M H ,et al. SmartEntry:mitigating routing update overhead with reinforcement learning for traffic engineering[C]// Proceedings of the Workshop on Network Meets AI ＆ ML. Piscataway:IEEE Press, 2020: 1-7.
[13]	HUANG X H , YUAN T T , QIAO G H ,et al. Deep reinforcement learning for multimedia traffic control in software defined networking[J]. IEEE Network, 2018,32(6): 35-41.
[14]	HUANG X H , ZENG M , XIE K . Intelligent traffic control for QoS optimization in hybrid SDNs[J]. Computer Networks, 2021,189: 107877.
[15]	ZHANG Q Y , WANG X W , LV J H ,et al. Intelligent content-aware traffic engineering for SDN:an AI-driven approach[J]. IEEE Network, 2020,34(3): 186-193.
[16]	XU Z Y , TANG J , MENG J S ,et al. Experience-driven networking:a deep reinforcement learning based approach[C]// Proceedings of IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2018: 1871-1879.
[17]	KATO N , FADLULLAH Z M , MAO B M ,et al. The deep learning vision for heterogeneous network traffic control:proposal,challenges,and future perspective[J]. IEEE Wireless Communications, 2017,24(3): 146-153.
[18]	ZHANG X C , WANG Y L , GENG G G ,et al. Delay-optimized multicast tree packing in software-defined networks[J]. IEEE Transactions on Services Computing, 2021,PP(99): 1.
[19]	WU T , ZHOU P , WANG B H ,et al. Joint traffic control and multi-channel reassignment for core backbone network in SDN-IoT:a multi-agent deep reinforcement learning approach[J]. IEEE Transactions on Network Science and Engineering, 2021,8(1): 231-245.
[20]	SAHA N , MISRA S , BERA S . Q-flag:QoS-aware flow-rule aggregation in software-defined IoT networks[J]. IEEE Internet of Things Journal, 2022,9(7): 4899-4906.
[21]	FILALI A , KOBBANE A , ELMACHKOUR M ,et al. SDN controller assignment and load balancing with minimum quota of processing capacity[C]// Proceedings of IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2018: 1-6.
[22]	SAVAS S S , TORNATORE M , DIKBIYIK F ,et al. RASCAR:recovery-aware switch-controller assignment and routing in SDN[J]. IEEE Transactions on Network and Service Management, 2018,15(4): 1222-1234.
[23]	WANG T , LIU F M , XU H . An efficient online algorithm for dynamic SDN controller assignment in data center networks[J]. IEEE/ACM Transactions on Networking, 2017,25(5): 2788-2801.
[24]	BERA S , MISRA S , SAHA N . Traffic-aware dynamic controller assignment in SDN[J]. IEEE Transactions on Communications, 2020,68(7): 4375-4382.
[25]	BOUZIDI E H , OUTTAGARTS A , LANGAR R ,et al. Dynamic clustering of software defined network switches and controller placement using deep reinforcement learning[J]. Computer Networks, 2022,207:108852.
[26]	LIN N , ZHAO Q , ZHAO L ,et al. A novel cost-effective controller placement scheme for software-defined vehicular networks[J]. IEEE Internet of Things Journal, 2021,8(18): 14080-14093.
[27]	MAITY I , MISRA S , MANDAL C . CORE:prediction-based control plane load reduction in software-defined IoT networks[J]. IEEE Transactions on Communications, 2021,69(3): 1835-1844.

Metrics

Recommended 0

No Suggested Reading articles found!

属性	配置
操作系统	Ubuntu 16
CPU	Intel(R) Core(TM) i5-10400F
内存	16.0 GB
Python	Ver.3.7
Tensorflow	Ver.1.14
Mininet	Ver.2.3.0
Ryu	Ver.4.34
Iperf	Ver.2.0.9

Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 27

Related Articles 15

Metrics

Recommended 0

[1]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[2]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[3]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[4]	Dongbin WANG, Dongzhe WU, Hui ZHI, Kun GUO, Xu ZHANG, Jinqiao SHI, Yu ZHANG, Yueming LU. Preventing flow table overflow against denial of service attack in software defined network [J]. Journal on Communications, 2023, 44(2): 1-11.
[5]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[6]	Changchuan YIN, Yanjue LI, Hailong ZHU, Xinxin HE, Wenxuan HAN. HSTC: hybrid traffic scheduling mechanism in time-sensitive networking [J]. Journal on Communications, 2022, 43(6): 119-132.
[7]	Binghao YAN, Qinrang LIU, Jianliang SHEN, Xiantuo TANG, Dong LIANG. Fast loop-free path migration strategy in software defined network [J]. Journal on Communications, 2022, 43(5): 24-35.
[8]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.
[9]	Chuanhuang LI, Yangting CHEN, Jingjing TANG, Jiali LOU, Renhua XIE, Chuntao FANG, Weiming WANG, Chao CHEN. QL-STCT: an intelligent routing convergence method for SDN link failure [J]. Journal on Communications, 2022, 43(2): 131-142.
[10]	Jinchuan PEI, Yuxiang HU, Le TIAN, Tao HU, Ziyong LI. Time-sensitive network traffic scheduling method with joint routing planning [J]. Journal on Communications, 2022, 43(12): 54-65.
[11]	Xin SU, Leilei MENG, Yiqing ZHOU, Wu CELIMUGE. Maritime mobile edge computing offloading method based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(10): 133-145.
[12]	Li’na DU, Li ZHUO, Shuo YANG, Jiafeng LI, Jing ZHANG. Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services [J]. Journal on Communications, 2021, 42(9): 205-217.
[13]	Wenchen HE, Shaoyong GUO, Xuesong QIU, Liandong CHEN, Suxiang ZHANG. Node selection method in federated learning based on deep reinforcement learning [J]. Journal on Communications, 2021, 42(6): 62-71.
[14]	Qizhao ZHOU, Junqing YU, Dong LI. Research on flood defense mechanism of SDN control layer:detection and mitigation [J]. Journal on Communications, 2021, 42(11): 41-53.
[15]	Shuopeng LI, Juan FANG, Ken CHEN. DetNet service share protection scheme based on SRv6 [J]. Journal on Communications, 2021, 42(10): 32-42.