基于强化学习的服务链映射算法

doi:10.11959/j.issn.1000-436x.2018002

Abstract

Abstract:

A service chain resource scheduling architecture of multi-agent based on artificial intelligence technology was proposed.Meanwhile,a service chain mapping algorithm based on reinforcement learning was designed.Through the Q-learning mechanism,the location of each virtual network element in the service chain was determined according to the system status and the reward and punishment feedback after the deployment.The experimental results show that compared with the classical algorithms,the algorithm effectively reduces the average transmission delay of the service and improves the load balance of the system.

Key words: network function virtualization, artificial intelligence, service chain, reinforcement learning

CLC Number:

TP302

Liang WEI,Tao HUANG,Jiao ZHANG,Zenan WANG,Jiang LIU,Yunjie LIU. Service chain mapping algorithm based on reinforcement learning[J]. Journal on Communications, 2018, 39(1): 90-100.

Figures/Tables 7

变量	意义
G = (V,E)	网络拓扑
v _i∈V	其中一个节点
e _i∈E	其中一条链路
$C_{v_{i}}$	服务器节点总的vCPU数量
$\tilde{C_{v_{i}}}$	服务器节点已经使用的vCPU数量
$D (v_{i}, v_{j})$	节点v_i和节点v_j之间的最短路径的链路延时
$F = (f_{1}, f_{2}, f_{3}, \dots)$	系统支持的VNF的类型
$S_{i} = (s_{i}_{, 1}, s_{i}_{, 2}, s_{i}_{, 3}, \dots)$	服务链请求，由不同的网络功能按次序组成
f(s_i,j)	网络功能s_i,j所需要的VNF类型
\|S_i\|	服务链的的长度
v(s_i,j)	s_i,j部署的服务器节点位置

References 24

[1]	HAN B , GOPALAKRISHNAN V , JI L ,et al. Network function virtualization:challenges and opportunities for innovations[J]. IEEE Communications Magazine, 2015,53(2): 90-97.
[2]	MIJUMBI R , SERRAT J , GORRICHO J L ,et al. Network function virtualization:state-of-the-art and research challenges[J]. IEEE Communications Surveys ＆ Tutorials, 2016,18(1): 236-262.
[3]	RIERA J F , ESCALONA E , BATALLé J ,et al. Virtual network function scheduling:concept and challenges[C]// 2014 International Conference on Smart Communications in Network Technologies (SaCoNeT). 2014: 1-5.
[4]	MIJUMBI R , SERRAT J , GORRICHO J L ,et al. Design and evaluation of algorithms for mapping and scheduling of virtual network functions[C]// 2015 1st IEEE Conference on Network Softwarization (NetSoft). 2015: 1-9.
[5]	KREUTZ D , RAMOS F M V , VERíSSIMO P E ,et al. Software-defined networking:a comprehensive survey[J]. Proceedings of the IEEE, 2015,103(1): 14-76.
[6]	NUNES B A A , MENDONCA M , NGUYEN X N ,et al. A survey of software-defined networking:past,present,and future of programmable networks[J]. IEEE Communications Surveys ＆ Tutorials, 2014,16(3): 1617-1634.
[7]	LI Y , CHEN M . Software-defined network function virtualization:a survey[J]. IEEE Access, 2015,3: 2542-2553.
[8]	BHAMARE D , JAIN R , SAMAKA M ,et al. A survey on service function chaining[J]. Journal of Network and Computer Applications, 2016,75: 138-155.
[9]	KUO T W , LIOU B H , LIN K C J ,et al. Deploying chains of virtual network functions:on the relation between link and server usage[C]// IEEE INFOCOM 2016-the 35th Annual IEEE International Conference on Computer Communications. 2016: 1-9.
[10]	MECHTRI M , GHRIBI C , ZEGHLACHE D . A scalable algorithm for the placement of service function chains[J]. IEEE Transactions on Network and Service Management, 2016,13(3): 533-546.
[11]	WANG L , LU Z , WEN X ,et al. Joint optimization of service function chaining and resource allocation in network function virtualization[J]. IEEE Access, 2016,4: 8084-8094.
[12]	YE Z , CAO X , WANG J ,et al. Joint topology design and mapping of service function chains for efficient,scalable,and reliable network functions virtualization[J]. IEEE Network, 2016,30(3): 81-87.
[13]	REDDY V S , BAUMGARTNER A , BAUSCHERT T . Robust embedding of VNF/service chains with delay bounds[C]// 2016 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN). 2016: 93-99.
[14]	MEHRAGHDAM S , KELLER M , KARL H . Specifying and placing chains of virtual network functions[C]// 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet). 2014: 7-13.
[15]	ZHANG Q , XIAO Y , LIU F ,et al. Joint optimization of chain placement and request scheduling for network function virtualization[C]// 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS). 2017: 731-741.
[16]	HIRWE A , KATAOKA K . LightChain:a lightweight optimisation of VNF placement for service chaining in NFV[C]// 2016 IEEE NetSoft Conference and Workshops (NetSoft). 2016: 33-37.
[17]	ZHANG B , ZHANG P , ZHAO Y . Co-scaler:cooperative scaling of software-defined NFV service function chain[C]// 2016 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN). 2016: 33-38.
[18]	SAHHAF S , TAVERNIER W , COLLE D ,et al. Network service chaining with efficient network function mapping based on service decompositions[C]// 2015 1st IEEE Conference on Network Softwarization (NetSoft). 2015: 1-5.
[19]	MOENS H , TURCK F D . Customizable function chains:managing service chain variability in hybrid NFV networks[J]. IEEE Transactions on Network and Service Management, 2016,13(4): 711-724.
[20]	FEI X , LIU F , XU H ,et al. Towards load-balanced VNF assignment in geo-distributed NFV Infrastructure[C]// IEEE/ACM 25th International Symposium on Quality of Service (IWQoS). 2017: 1-10.
[21]	DIETRICH D , ABUJODA A , RIZK A ,et al. Multi-provider service chain embedding with nestor[J]. IEEE Transactions on Network and Service Management, 2017,14(1): 91-105.
[22]	HABIB A , KHAN M I . Reinforcement learning based autonomic virtual machine management in clouds[C]// 2016 5th International Conference on Informatics,Electronics and Vision (ICIEV). 2016: 1083-1088.
[23]	GROLéAT T , POUYLLAU H . Distributed inter-domain SLA negotiation using Reinforcement Learning[C]// 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011)and Workshops. 2011: 33-40.
[24]	LIN S C , AKYILDIZ I F , WANG P ,et al. QoS-aware adaptive routing in multi-layer hierarchical software defined networks:a reinforcement learning approach[C]// 2016 IEEE International Conference on Services Computing (SCC). 2016: 25-33.

Metrics

Recommended 0

No Suggested Reading articles found!

Service chain mapping algorithm based on reinforcement learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 24

Related Articles 15

Metrics

Recommended 0

[1]	Ling MA, Qiliang FAN, Ting XU, Guanchen GUO, Shenglin ZHANG, Yongqian SUN, Yuzhi ZHANG. Scheduling framework based on reinforcement learning in online-offline colocated cloud environment [J]. Journal on Communications, 2023, 44(6): 90-102.
[2]	Biao JIN, Yikang LI, Zhiqiang YAO, Yulin CHEN, Jinbo XIONG. GenFedRL: a general federated reinforcement learning framework for deep reinforcement learning agents [J]. Journal on Communications, 2023, 44(6): 183-197.
[3]	Yuancheng LI, Yongtai QIN. Deep reinforcement learning based algorithm for real-time QoS optimization of software-defined security middle platform [J]. Journal on Communications, 2023, 44(5): 181-192.
[4]	Dacheng ZHOU, Hongchang CHEN, Weizhen HE, Guozhen CHENG, Hongchao HU. Research on multidimensional dynamic defense strategy for microservice based on deep reinforcement learning [J]. Journal on Communications, 2023, 44(4): 50-63.
[5]	Guoliang XU, Feng TAN, Yongyi RAN, Feng CHEN. Joint beam hopping and coverage control optimization algorithm for multibeam satellite system [J]. Journal on Communications, 2023, 44(4): 78-86.
[6]	Feibo JIANG, Yubo PENG, Li DONG. Deep image semantic communication model for 6G [J]. Journal on Communications, 2023, 44(3): 198-208.
[7]	Wenjun XU, Silei WU, Fengyu WANG, Lan LIN, Guojun LI, Zhi ZHANG. Large-scale post-disaster user distributed coverage optimization based on multi-agent reinforcement learning [J]. Journal on Communications, 2022, 43(8): 1-16.
[8]	Zongxuan SHA, Ru HUO, Chuang SUN, Shuo WANG, Tao HUANG. Forwarding efficiency aware traffic scheduling algorithm based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 30-40.
[9]	Zexi XU, Lei ZHUANG, Kunli ZHANG, Mingyu GUI. Online placement algorithm of service function chain based on knowledge graph [J]. Journal on Communications, 2022, 43(8): 41-51.
[10]	Shuai MA, Bing LI, Haihong SHENG, Rongyan GU, Hui ZHOU, Hongmei WANG, Yue WANG, Shiyin LI. Research on power allocation of integrated VLPC based on deep reinforcement learning [J]. Journal on Communications, 2022, 43(8): 121-130.
[11]	Yu ZHANG, Min CHENG. Joint optimization of edge computing and caching in NDN [J]. Journal on Communications, 2022, 43(8): 164-175.
[12]	Peiliang ZUO, Shaolong HOU, Chao GUO, Hua JIANG, Wenbo WANG. Security decision method for the edge of multi-layer satellite network based on reinforcement learning [J]. Journal on Communications, 2022, 43(6): 189-199.
[13]	Jingyu WANG, Zirui ZHUANG. Research on a knowledge-defined polymorphic network attainable service architecture [J]. Journal on Communications, 2022, 43(4): 71-82.
[14]	Zhisheng NIU, Sheng ZHOU, Yuxuan SUN. Green communication and networking for Carbon-peaking and Carbon-neutrality: challenges and solutions [J]. Journal on Communications, 2022, 43(2): 1-14.
[15]	Xianchao ZHANG, Yao ZHAO, Haijun YE, Rui FAN. Intelligent transmit power control algorithm for the multi-user interference of wireless network [J]. Journal on Communications, 2022, 43(2): 15-21.