基于Q-learning算法的vEPC虚拟网络功能部署方法

doi:10.11959/j.issn.1000-436x.2017173

Abstract

Abstract:

In the context of vEPC，a method of virtua1ized network function（VNF）dep1oyment via an improved Q-1earning a1gorithm was proposed to so1ve the prob1em that the existing methods cannot achieve the optimization of time de1ay and revenue of VNF dep1oyment simu1taneous1y.To get the optima1 dep1oyment po1icy in both space dimension and time dimension，a Markov decision process mode1 of vEPC service function chain dep1oyment on the basis of the traditiona1 0-1 programming mode1 was estab1ished and a so1ution with an improved Q-1earning a1gorithm was proposed.The method had taken fu11 consideration of both virtua1 network embedding in space dimension and orchestration of VNF 1ife cyc1e in time dimension，and thus，the mu1ti-objective optimization of revenue and de1ay cou1d be attained.Simu1ation shows that the method can reduce network de1ay whi1e increasing the revenue and the ratio of request acceptance compared with other dep1oyment methods.

Key words: 5G, VNF, service function chain dep1oyment, Q-1earning

CLC Number:

TN915.81

Quan YUAN,Hong-bo TANG,Kai-zhi HUANG,Xiao-lei WANG,Yu ZHAO. Deployment method for vEPC virtualized network function via Q-learning[J]. Journal on Communications, 2017, 38(8): 172-182.

Figures/Tables 15

参数	定义
m	租户请求的最大VNF数量
n	底层网络中物理节点的总数量
k	表示底层网络中提供的物理资源类型
l	服务链中请求的VNF种类
$C_{n \times k}$	底层网络资源容量矩阵
$R_{m \times k}$	VNF服务链请求的资源矩阵
$B_{n \times n}$	物理节点的邻接矩阵
$V_{m \times m}$	VNF邻接矩阵
$W_{n \times n}$	传输时延开销矩阵
$W_{1 \times l}^{p}$	处理时延开销向量
$L_{1 \times m}$	服务链的VNF组成向量
$E_{1 \times m}$	服务链的部署收益向量
$X_{m \times n}$	服务链的部署关系矩阵

References 17

[1]	SAMA M R , CONTRERAS L M , KAIPPALLIMALIL J ,et al. Software-defined control of the virtualized mobile packet core[J]. IEEE Communications Magazine, 2015,53(2): 107-115.
[2]	TALEB T , CORICI M , PARADA C ,et al. EASE:EPC as a service to ease mobile core network deployment over cloud[J]. IEEE Network, 2015,29(2): 78-88.
[3]	MOENS H , DE F . VNF-P:a model for efficient placement of virtualized network functions[C]// IEEE International Conference on Network and Service Management. 2014: 418-423.
[4]	CLAYMAN S , MAINI E , GALIS A ,et al. The dynamic placement of virtual network functions[C]// IEEE International Conference on Network Operations and Management Symposium. 2014: 1-9.
[5]	BASTA A , KELLERER W , HOFFMANN M ,et al. Applying NFV and SDN to LTE mobile core gateways,the functions placement problem[C]// ACM Workshop on All Things Cellular:Operations,Applications,＆ Challenges, 2014: 33-38.
[6]	江逸茗, 兰巨龙, 周惠琴 . 网络虚拟化环境下的资源监控策略[J]. 电子与信息学报, 2014,36(3): 708-714.
	JIANG Y M , LAN J L , ZHOU H Q . Resource monitoring policy for network virtualization environment[J]. Journal of Electronics ＆ Information Technology, 2014,36(3): 708-714.
[7]	MIJUMBI R , SERRAT J , GORRICHO J L ,et al. Design and evaluation of algorithms for mapping and scheduling of virtual network functions[C]// IEEE Conference on Network Softwarization. 2015: 1-9.
[8]	HAN B , GOPALAKRISHNAN V , JI L S ,et al. Network function virtualization:challenges and opportunities for innovations[J]. IEEE Communications Magazine, 2015,53(2): 90-97.
[9]	BASTA A , KELLERER W , HOFFMANN M ,et al. A virtual SDN-enabled LTE EPC architecture:a case study for S-/P-Gateways functions[C]// IEEE International Conference on SDN for Future Networks and Services. 2013: 1-7.
[10]	FISCHER A , BOTERO J F , TILL BECK M ,et al. Virtual network embedding:a survey[J]. IEEE Communications Surveys ＆ Tutorials, 2013,15(4): 1888-1906.
[11]	ROSS S M . Stochastic processes[J]. John Wiley ＆ Sons Inc New York, 1996,48(1): 528-529.
[12]	SONG H , LIU C , LAWARRéE J ,et al. Optimal electricity supply bidding by Markov decision process[J]. IEEE Transactions on Power Systems, 2000,15(2): 618-624.
[13]	WATKINS C , DAYAN P . Q-learning[J]. Machine Learning, 1992,8(3-4): 279-292.
[14]	BUSONIU L , BABUSKA R . Reinforcement learning and dynamic programming using function approximators[M]. Florida: CRC PressPress, 2010.
[15]	DUONG T , CHU Y , NGUYEN T ,et al. Virtual machine placement via Q-learning with function approximation[C]// 2015 IEEE Global Communications Conference. 2015: 1-6.
[16]	ZEGURA E W , CALVERT K L , ACHARJEE S B . How to model an internetwork[C]// IEEE Conference of Computer Societies,Networking the Next Generation. 1996: 594-602.
[17]	ORLOWSKI S , WESS?LY R , PIóRO M ,et al. 0-survivable network design library[J]. Networks, 2010,55(3): 276-286.

Metrics

Recommended 0

No Suggested Reading articles found!

Deployment method for vEPC virtualized network function via Q-learning

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 17

Related Articles 15

Metrics

Recommended 0

[1]	Caixia LIU, Xinsheng JI, Jiangxing WU. Endogenous security common problems and solutions of the mobile communication networks [J]. Journal on Communications, 2022, 43(9): 70-79.
[2]	Hang QIU, Hongbo TANG, Wei YOU, Yu ZHAO, Yi BAI. QGA-based network service extension algorithm in NFV [J]. Journal on Communications, 2022, 43(11): 44-52.
[3]	Huiqiang WANG, Kaixuan GAO, Hongwu LYU. Survey of high-precision localization and the prospect of future evolution [J]. Journal on Communications, 2021, 42(7): 198-210.
[4]	Lei SUN, Jianquan WANG, Shangjing LIN, Zhangchao MA, Wei LI, Liang Qilian, Rong HUANG. Research on 5G-TSN joint scheduling mechanism based on radio channel information [J]. Journal on Communications, 2021, 42(12): 65-75.
[5]	Yueping CAI, Dong LI, Chi XU, Zhen WANG, Xiaowen ZHANG. Integrating 5G-U with time-sensitive networking for industrial Internet: architectures and technologies [J]. Journal on Communications, 2021, 42(10): 43-54.
[6]	Yusun FU,Genke YANG. Wireless ultra-reliable and low-latency communication:key design analysis and challenge [J]. Journal on Communications, 2020, 41(8): 187-203.
[7]	Xiaona LI,Zhongfang WANG,Qian CHENG,Jingwen FU,Shunliang ZHANG. Dynamic interference coordination scheme based on the iterative joint transmission in ultra dense network [J]. Journal on Communications, 2020, 41(2): 176-186.
[8]	. Challenges for beyond 5G:ultra-densification of radio access network [J]. Journal on Communications, 2020, 41(11): 1-11.
[9]	Wenxue LIU,Shijun CHEN,Jian GE,Hong YUAN,Cuiling GONG. Research on nanosecond time synchronization technology for 5G base station based on GNSS neighborhood similarity [J]. Journal on Communications, 2020, 41(1): 180-190.
[10]	Xutong ZUO,Mowei WANG,Yong CUI. Low-latency networking:architecture,key scenarios and research prospect [J]. Journal on Communications, 2019, 40(8): 22-35.
[11]	Yuan YANG,Mingwei XU,Hao CHEN. Analysis and modeling of Internet backbone traffic with 5G/B5G [J]. Journal on Communications, 2019, 40(8): 36-44.
[12]	HUANG Tao,WANG Shuo,HUANG Yudong,ZHENG Yao,LIU Jiang,LIU Yunjie. Survey of the deterministic network [J]. Journal on Communications, 2019, 40(6): 160-176.
[13]	Wenjuan SHAO,Qingguo SHEN. Survey of software defined D2D and V2X communication [J]. Journal on Communications, 2019, 40(4): 179-194.
[14]	Zheng HU,Hao YUAN,Xinning ZHU,Wanli NI. Research on crowd flows prediction model for 5G demand [J]. Journal on Communications, 2019, 40(2): 1-10.
[15]	Guangyi LIU,Qixing WANG,Jianjun LIU,Fei WANG,Yi ZHENG. Latest progress on 3.5 GHz 5G NR trial [J]. Journal on Communications, 2019, 40(2): 24-30.