低时延网络：架构，关键场景与研究展望

doi:10.11959/j.issn.1000-436x.2019175

摘要/Abstract

摘要：

随着时延敏感型应用和超低时延场景的出现，低时延网络的研究受到了学术界、工业界和标准组织的广泛关注。了解时延产生的原因并设计相应的降低时延的技术使新兴应用的发展成为可能。首先按照网络的分层体系架构对时延来源进行分析，并以此为基础对降低时延的技术进行了综述。然后，针对数据中心网络、5G 以及边缘计算3种典型的低时延关键场景及优化时延的技术进行分析。最后，从网络架构革新、数据驱动优化时延算法及新协议设计3个方面展望了低时延网络发展面临的机遇与挑战。

关键词: 低时延, 网络架构, 时延来源, 数据中心网络, 第五代移动通信技术, 边缘计算

Abstract:

With the advent of delay-sensitive applications and ultra-low latency scenarios,research on low-latency networking is attracting attention from academia,industry,and standards organizations.Understanding the causes of latency and designing corresponding techniques to reduce latency enable the development of emerging applications.The sources of latency according to the layered architecture of the network was analyzed,and summarizes the techniques for reducing the latency.After that,three typical low-latency key scenarios and delay optimization techniques for data center network,5G and edge computing was analyzed.Finally,the opportunities and challenges that may be encountered in the development of low latency networks were presented from the perspectives of network architecture innovation,data-driven latency optimization algorithm and the design of new protocols.

Key words: low latency, network architecture, causes of latency, data center network, 5G, edge computing

中图分类号:

TP393

左旭彤,王莫为,崔勇. 低时延网络：架构，关键场景与研究展望[J]. 通信学报, 2019, 40(8): 22-35.

Xutong ZUO,Mowei WANG,Yong CUI. Low-latency networking:architecture,key scenarios and research prospect[J]. Journal on Communications, 2019, 40(8): 22-35.

图/表 6

表1

图1

图2

图3

图4

图5

参考文献 102

[1]	BRISCOE B , BRUNSTROM A , PETLUND A ,et al. Reducing internet latency:a survey of techniques and their merits[J]. IEEE Communications Surveys ＆ Tutorials, 2016,18(3): 2149-2196.
[2]	ZUO X , CUI Y , WANG M ,et al. Low-latency networking:architecture,techniques,and opportunities[J]. IEEE Internet Computing, 2018,22(5): 56-63.
[3]	ZHANG J , REN F , LIN C . Survey on transport control in data center networks[J]. IEEE Network, 2013,27(4): 22-26.
[4]	XIA W , ZHAO P , WEN Y ,et al. A survey on data center networking (DCN):infrastructure and operations[J]. IEEE communications surveys ＆ tutorials, 2017,19(1): 640-656.
[5]	张平, 陶运铮, 张治 . 5G 若干关键技术评述[J]. 通信学报, 2016,37(7): 15-29.
	ZHANG P , TAO Y Z , ZHANG Z . Survey of several key technologies for 5G[J]. Journal on Communications, 2016,37(7): 15-29.
[6]	GUPTA A , JHA R K . A survey of 5G network:architecture and emerging technologies[J]. IEEE access, 2015(3): 1206-1232.
[7]	MAO Y , YOU C , ZHANG J ,et al. A survey on mobile edge computing:the communication perspective[J]. IEEE Communications Surveys ＆ Tutorials, 2017,19(4): 2322-2358.
[8]	MACH P , BECVAR Z . Mobile edge computing:a survey on architecture and computation offloading[J]. IEEE Communications Surveys ＆Tutorials, 2017,19(3): 1628-1656.
[9]	PANG H , ZHANG C , WANG F ,et al. Optimizing personalized interaction experience in crowd-interactive livecast:a cloud-edge approach[C]// ACM Multimedia Conference on Multimedia Conference. ACM, 2018: 1217-1225.
[10]	LAI Z , HU Y C , CUI Y ,et al. Furion:engineering high-quality immersive virtual reality on today's mobile devices[C]// The 23rd Annual International Conference on Mobile Computing and Networking. ACM, 2017: 409-421.
[11]	SAMII S , ZINNER H . Level 5 by layer 2:time-sensitive networking for autonomous vehicles[J]. IEEE Communications Standards Magazine, 2018,2(2): 62-68.
[12]	FETTWEIS G , BOCHE H , WIEGAND T ,et al. The tactile internet-ITU-T technology watch report[J]. Geneva:ITU, 2014
[13]	WANG B , ZHANG X , WANG G ,et al. Anatomy of a personalized livestreaming system[C]// The 2016 Internet Measurement Conference. ACM, 2016: 485-498.
[14]	LIANG G , LIANG B . Balancing interruption frequency and buffering penalties in VBR video streaming[C]// IEEE International Conference on Computer Communications. IEEE, 2007: 1406-1414.
[15]	WANG J , LEI W , XU P ,et al. Adaptive media playout buffer management for latency optimization of mobile live streaming[C]// IEEE International Conference on Multimedia ＆ Expo Workshops. IEEE, 2017: 369-374.
[16]	CHENG Y , CHU J , RADHAKRISHNAN S ,et al. TCP fast open:RFC 7413[S].(2014-10)[2019-05-23].
[17]	CUI Y , LI T , LIU C ,et al. Innovating transport with QUIC:design approaches and research challenges[J]. IEEE Internet Computing, 2017,21(2): 72-76.
[18]	ALLMAN M , FLOYD S , PARTRIDGE C . Increasing TCP's initial window:RFC 3390[S].(2002-10)[2019-05-23].
[19]	DUKKIPATI N , REFICE T , CHENG Y ,et al. An argument for increasing TCP's initial congestion window[J]. ACM Sigcomm Computer Communication Review, 2010,40(3): 26-33.
[20]	WANG R , PAU G , YAMADA K ,et al. TCP startup performance in large bandwidth networks[C]// IEEE International Conference on Computer Communications. IEEE, 2004(2): 796-805.
[21]	STEVENS W R . TCP slow start,congestion avoidance,fast retransmit,and fast recovery algorithms:RFC 2001[S].(1997-02)[2019-05-23].
[22]	CHENG P , REN F , SHU R ,et al. Catch the whole lot in an action:rapid precise packet loss notification in data center[C]// USENIX Conference on Networked Systems Design and Implementation. USENIX Association, 2014: 17-28.
[23]	SHI H , CUI Y , WANG X ,et al. STMS:improving MPTCP throughput under heterogeneous networks[C]// USENIX Annual Technical Conference. USENIX, 2018: 719-730.
[24]	PAASCH C , FERLIN S , ALAY O ,et al. Experimental evaluation of multipath TCP schedulers[C]// The ACM SIGCOMM workshop on Capacity sharing workshop. ACM, 2014: 27-32.
[25]	LANGLEY A , RIDDOCH A , WILK A ,et al. The quic transport protocol:design and internet-scale deployment[C]// The Conference of the ACM Special Interest Group on Data Communication. ACM, 2017: 183-196.
[26]	BISHOP M . Hypertext transfer protocol version 3 (HTTP/3)[S].(2019-07-09)[2019-05-23].
[27]	TAN K , SONG J , ZHANG Q ,et al. A compound TCP approach for high-speed and long-distance networks[C]// 25th IEEE International Conference on Computer Communications. IEEE, 2006: 1-12.
[28]	ARUN V , BALAKRISHNAN H . Copa:practical delay-based congestion control for the Internet[C]// 15th Symposium on Networked Systems Design and Implementation. ACM, 2018: 329-342.
[29]	CARLUCCI G , DE CICCO L , HOLMER S ,et al. Analysis and design of the Google congestion control for Web real-time communication (WebRTC)[C]// The 7th International Conference on Multimedia Systems. ACM, 2016:13.
[30]	TASSIULAS L , EPHREMIDES A . Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multi-hop radio networks[C]// 29th IEEE Conference on Decision and Control. IEEE, 1990: 2130-2132.
[31]	BUI L X , SRIKANT R , STOLYAR A . A novel architecture for reduction of delay and queueing structure complexity in the back-pressure algorithm[J]. IEEE/ACM Transactions on Networking, 2011,19(6): 1597-1609.
[32]	WANG N , HO K H , PAVLOU G ,et al. An overview of routing optimization for internet traffic engineering[J]. IEEE Communications Surveys ＆ Tutorials, 2008,10(1): 36-56.
[33]	AWDUCHE D , MALCOLM J , AGOGBUA J ,et al. Requirements for traffic engineering over MPLS:RFC 2702[S].(1999-09)[2019-05-23].
[34]	FORTZ B , THORUP M . Internet traffic engineering by optimizing OSPF weights[C]// IEEE Conference on Computer Communications,Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. IEEE, 2000(2): 519-528.
[35]	HARMATOS J , . A heuristic algorithm for solving the static weight optimisation problem in OSPF networks[C]// IEEE Global Telecommunications Conference. IEEE, 2001(3): 1605-1609.
[36]	SRIDHARAN A , GUéRIN R , DIOT C . Achieving near-optimal traffic engineering solutions for current OSPF/IS-IS networks[J]. IEEE/ACM Transactions On Networking, 2005,13(2): 234-247.
[37]	XU D , CHIANG M , REXFORD J . Link-state routing with hop-by-hop forwarding can achieve optimal traffic engineering[J]. IEEE/ACM Transactions on networking, 2011,19(6): 1717-1730.
[38]	AL-FARES M , RADHAKRISHNAN S , RAGHAVAN B ,et al. Hedera:dynamic flow scheduling for data center networks[C]// The 7^th USENIX Symposium on Networked Systems Design and Implementation. USENIX, 2010: 281-295.
[39]	GVOZDIEV N , VISSICCHIO S , KARP B ,et al. On low-latency-capable topologies,and their impact on the design of intra-domain routing[C]// The 2018 Conference of the ACM Special Interest Group on Data Communication. ACM, 2018: 88-102.
[40]	SKORDOULIS D , NI Q , CHEN H H ,et al. IEEE 802.11 n MAC frame aggregation mechanisms for next-generation high-throughput WLANs[J]. IEEE Wireless Communications, 2008,15(1): 40-47.
[41]	LóPEZ-PéREZ D , GARCIA-RODRIGUEZ A , GALATIGIORDANO L ,et al. IEEE 802.11 be - extremely high throughput:the next generation of Wi-Fi technology beyond 802.11 ax[J]. Cornell University:arXiv:1902.04320, 2019.
[42]	IEEE. IEEE standard for ethernet amendment 5:specification and management parameters for interspersing express traffic:802.3br2016[S]. 2016. 1-58.
[43]	IEEE. IEEE standard for local and metropolitan area networks—bridges and bridged networks—amendment 26:frame preemption:802.1Qbu-2016[S]. 2016. 1-52.
[44]	IEEE. IEEE Standard for local and metropolitan area networks—bridges and bridged networks—amendment 25:enhancements for scheduled traffic:802.1Qbv-2015[S]. 2016. 1-57.
[45]	PAXSON V . On calibrating measurements of packet transit times[J]. ACM Sigmetrics Performance Evaluation Review.ACM, 1998,26(1): 11-21.
[46]	MOON S B , SKELLY P , TOWSLEY D . Estimation and removal of clock skew from network delay measurements[C]// IEEE Conference on Computer Communications,Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. IEEE, 1999(1): 227-234.
[47]	STROWES S D . Passively measuring TCP round-trip times[J]. Communications of the ACM, 2013,56(10): 57-64.
[48]	CARRA D , AVRACHENKOV K , ALOUF S ,et al. Passive online RTT estimation for flow-aware routers using one-way traffic[C]// International Conference on Research in Networking. Springer, 2010: 109-121.
[49]	DE VAERE P , BüHLER T , KüHLEWIND M ,et al. Three bits suffice:explicit support for passive measurement of internet latency in QUIC and TCP[C]// The Internet Measurement Conference. ACM, 2018: 22-28.
[50]	ALIZADEH M , GREENBERG A , MALTZ D A ,et al. Data center TCP (DCTCP)[J]. ACM SIGCOMM computer communication review, 2011,41(4): 63-74.
[51]	ALIZADEH M , YANG S , SHARIF M ,et al. PFABRIC:minimal near-optimal datacenter transport[C]// ACM SIGCOMM Computer Communication Review. ACM, 201343(4): 435-446.
[52]	GUO C , WU H , DENG Z ,et al. RDMA over commodity ethernet at scale[C]// The 2016 ACM SIGCOMM Conference. ACM, 2016: 202-215.
[53]	ZHU Y , ERAN H , FIRESTONE D ,et al. Congestion control for large-scale RDMA deployments[J]. ACM SIGCOMM Computer Communication Review, 2015,45(4): 523-536.
[54]	AL-FARES M , LOUKISSAS A , VAHDAT A . A scalable,commodity data center network architecture[C]// ACM SIGCOMM Conference on Applications,Technologies,Architectures,and Protocols for Computer Communications. ACM, 2008: 63-74.
[55]	GREENBERG A , HAMILTON J R , JAIN N ,et al. VL2:a scalable and flexible data center network[J]. Communication of the ACM, 2009,39(4): 51-62.
[56]	GUO C , WU H , TAN K ,et al. Dcell:a scalable and fault-tolerant network structure for data centers[J]. ACM SIGCOMM Computer Communication Review.ACM, 2008,38(4): 75-86.
[57]	GUO C , LU G , LI D ,et al. BCube:a high performance,server-centric network architecture for modular data centers[J]. ACM SIGCOMM Computer Communication Review, 2009,39(4): 63-74.
[58]	SHIN J Y , WONG B , SIRER E G . Small-world datacenters[C]// The 2^nd ACM Symposium on Cloud Computing. ACM, 2011 2:1-2:13.
[59]	SINGLA A , HONG C Y , POPA L ,et al. Jellyfish:networking data centers randomly[C]// The 9th Symposium on Networked Systems Design and Implementation. 2012: 225-238.
[60]	CUI Y , XIAO S , WANG X ,et al. Diamond:nesting the data center network with wireless rings in 3-D space[J]. IEEE/ACM Transactions on Networking, 2018,26(1): 145-160.
[61]	WANG G , ANDERSEN D G , KAMINSKY M ,et al. C-through:part-time optics in data centers[J]. ACM SIGCOMM Computer Communication Review, 2011,41(4): 327-338.
[62]	FARRINGTON N , PORTER G , RADHAKRISHNAN S ,et al. Helios:a hybrid electrical/optical switch architecture for modular data centers[J]. ACM SIGCOMM Computer Communication Review, 2011,41(4): 339-350.
[63]	LEGTCHENKO S , CHEN N , CLETHEROE D ,et al. XFabric:a reconfigurable in-rack network for rack-scale computers[C]// The 13th Symposium on Networked Systems Design and Implementation. USENIX, 2016: 15-29.
[64]	HALPERIN D , KANDULA S , PADHYE J ,et al. Augmenting data center networks with multi-gigabit wireless links[J]. ACM SIGCOMM Computer Communication Review, 2011,41(4): 38-49.
[65]	ZHOU X , ZHANG Z , ZHU Y ,et al. Mirror mirror on the ceiling:flexible wireless links for data centers[J]. ACM SIGCOMM Computer Communication Review, 2012,42(4): 443-454.
[66]	HAMEDAZIMI N , QAZI Z , GUPTA H ,et al. Firefly:a reconfigurable wireless data center fabric using free-space optics[C]// ACM SIGCOMM Computer Communication Review, 201441(4): 319-330.
[67]	GHOBADI M , MAHAJAN R , PHANISHAYEE A ,et al. Projector:agile reconfigurable data center interconnect[C]// The ACM SIGCOMM Conference. ACM, 2016: 216-229.
[68]	WANG M , CUI Y , XIAO S ,et al. Neural network meets DCN:traffic-driven topology adaptation with deep learning[J]. The ACM on Measurement and Analysis of Computing Systems, 2018,pp. 1-25.
[69]	INTELLIGENCE G . Understanding 5G:perspectives on future technological advancements in mobile[R]. 2014: 1-26.
[70]	CISCO. Direct tunnel for 3G networks[EB].(2016-10-27)[2019-05-23]
[71]	IMT-2020(5G) 推进组. 5G 网络架构设计白皮书[R]. 2016.
	IMT-2020(5G) 推进组. White paper of 5G network architecture design[R]. 2016.
[72]	ANDREWS J G , BUZZI S , CHOI W ,et al. What will 5G be?[J]. IEEE Journal on selected areas in communications, 2014,32(6): 1065-1082.
[73]	RAPPAPORT T S , SUN S , MAYZUS R ,et al. Millimeter wave mobile communications for 5G cellular:it will work![J]. IEEE access, 2013,1: 335-349.
[74]	AGIWAL M , ROY A , SAXENA N . Next generation 5G wireless networks:a comprehensive survey[J]. IEEE Communications Surveys＆ Tutorials, 2016,18(3): 1617-1655.
[75]	PI Z , KHAN F . System design and network architecture for a millimeter-wave mobile broadband (MMB) system[C]// The 34th IEEE Sarnoff Symposium. IEEE, 2011: 1-6.
[76]	SAITO Y , KISHIYAMA Y , BENJEBBOUR A ,et al. Non-orthogonal multiple access (NOMA) for cellular future radio access[C]// IEEE 77th vehicular technology conference (VTC Spring). IEEE, 2013: 1-5.
[77]	NIKOPOUR H , BALIGH H . Sparse code multiple access[C]// 2013 IEEE 24th Annual International Symposium on Personal,Indoor,and Mobile Radio Communications. IEEE, 2013: 332-336.
[78]	ZHANG Y , LIU H , JIAO L ,et al. To offload or not to offload:an efficient code partition algorithm for mobile cloud computing[C]// IEEE 1st International Conference on Cloud Networking. IEEE, 2012: 80-86.
[79]	CUERVO E , BALASUBRAMANIAN A , CHO D ,et al. MAUI:making smartphones last longer with code offload[C]// The 8th international conference on Mobile systems,applications,and services. ACM, 2010: 49-62.
[80]	CHUN B G , IHM S , MANIATIS P ,et al. Clonecloud:elastic execution between mobile device and cloud[C]// The sixth Conference on Computer Systems. ACM, 2011: 301-314.
[81]	RA M R , SHETH A , MUMMERT L ,et al. Odessa:enabling interactive perception applications on mobile devices[C]// The 9th International Conference on Mobile Systems,Applications,and Services. ACM, 2011: 43-56.
[82]	KAO Y H , KRISHNAMACHARI B , RA M R ,et al. Hermes:Latency optimal task assignment for resource-constrained mobile computing[J]. IEEE Transactions on Mobile Computing, 2017,16(11): 3056-3069.
[83]	ANANTHANARAYANAN G , BAHL P , BODíK P ,et al. Real-time video analytics:the killer app for edge computing[J]. Computer, 2017,50(10): 58-67.
[84]	ZHANG H , ANANTHANARAYANAN G , BODIK P ,et al. Live video analytics at scale with approximation and delay-tolerance[C]// The 14th Symposium on Networked Systems Design and Implementation. USENIX, 2017: 377-392.
[85]	YI S , HAO Z , ZHANG Q ,et al. Lavea:latency-aware video analytics on edge computing platform[C]// The Second ACM/IEEE Symposium on Edge Computing. ACM, 2017(15): 1-13.
[86]	HONG C Y , KANDULA S , MAHAJAN R ,et al. Achieving high utilization with software-driven WAN[C]// ACM SIGCOMM Computer Communication Review. ACM, 2013,43(4): 15-26.
[87]	JAIN S , KUMAR A , MANDAL S ,et al. B4:experience with a globally-deployed software defined WAN[C]// ACM SIGCOMM Computer Communication Review. ACM, 2013,43(4): 3-14.
[88]	SINGH A , ONG J , AGARWAL A ,et al. Jupiter rising:a decade of clos topologies and centralized control in Google's datacenter network[J]. ACM SIGCOMM computer communication review, 2015,45(4): 183-197.
[89]	YAP K K , MOTIWALA M , RAHE J ,et al. Taking the edge off with espresso:Scale,reliability and programmability for global internet peering[C]// The Conference of the ACM Special Interest Group on Data Communication. ACM, 2017: 432-445.
[90]	DALTON M , SCHULTZ D , ADRIAENS J ,et al. Andromeda:performance,isolation,and velocity at scale in cloud network virtualization[C]// The 15^th Symposium on Networked Systems Design and Implementation. USENIX, 2018: 373-387.
[91]	KUMAR S , TUFAIL M , MAJEE S ,et al. Service function chaining use cases in data centers[J]. IETF, 2017.
[92]	SUN C , BI J , ZHENG Z ,et al. NFP:enabling network function parallelism in NFV[C]// The Conference of the ACM Special Interest Group on Data Communication. ACM, 2017: 43-56.
[93]	WANG M , CUI Y , WANG X ,et al. Machine learning for networking:Workflow,advances and opportunities[J]. IEEE Network, 2018,32(2): 92-99.
[94]	JIANG J , SEKAR V , STOICA I ,et al. Unleashing the potential of data-driven networking[C]// International Conference on Communication Systems and Networks. Springer, 2017: 110-126.
[95]	JIANG J , DAS R , ANANTHANARAYANAN G ,et al. Via:improving internet telephony call quality using predictive relay selection[C]// The 2016 ACM SIGCOMM Conference. ACM, 2016: 286-299.
[96]	WINSTEIN K , BALAKRISHNAN H . TCP ex machina:computer-generated congestion control[C]// The SIGCOMM Conference. ACM, 2013: 123-134.
[97]	DONG M , LI Q , ZARCHY D ,et al. PCC:re-architecting congestion control for consistent high performance[C]// The 12^th Symposium on Networked Systems Design and Implementation. USENIX, 2015: 395-408.
[98]	DONG M , MENG T , ZARCHY D ,et al. PCC vivace:online-learning congestion control[C]// The 15th USENIX Symposium on Networked Systems Design and Implementation. USENIX Association, 2018: 343-356.
[99]	CHEN L , LINGYS J , CHEN K ,et al. AuTO:scaling deep reinforcement learning to enable datacenter-scale automatic traffic optimization[C]// The SIGCOMM Conference. ACM, 2018: 191-205.
[100]	BELSHE M , PEON R , THOMSON M . Hypertext transfer protocol version 2 (Http/2):RFC 7540[S]. 2015:
[101]	HOLMBERG C , HAKANSSON S , ERIKSSON G . Web real-time communication use cases and requirements[J]. RFC 7478, 2015.
[102]	INFOQ. 快手多媒体传输算法优化实践[EB].(2019-01-09)[2019-05-23].
	INFOQ. 快手多媒体传输算法优化实践[EB].(2019-01-09)[2019-05-23].

应用/场景	时延需求	需求原因
直播^[9]	<1 s	人与人互动
VR/AR^[10]	10 ms	人机交互
自动驾驶^[11]	1~10 ms	快速应对环境变化人机交互
物联网/工业互联网^[12]	1~10 ms	设备与设备通信及协调工作