基于Telemetry架构的数据中心网络纳秒级时间同步

doi:10.11959/j.issn.1000-436x.2021147

通信学报 ›› 2021, Vol. 42 ›› Issue (10): 117-129.doi: 10.11959/j.issn.1000-436x.2021147

基于Telemetry架构的数据中心网络纳秒级时间同步

张千里, 张超凡, 王继龙, 唐翔宇, 沈钲晨, 王会

清华大学网络科学与网络空间研究院，北京 100084

修回日期:2021-07-29 出版日期:2021-10-25 发布日期:2021-10-01
作者简介:张千里（1975- ），男，内蒙古包头人，博士，清华大学副研究员，主要研究方向为下一代互联网体系结构、网络安全等
张超凡（1998- ），男，山东聊城人，清华大学硕士生，主要研究方向为可编程数据平面、时间同步等
王继龙（1973- ），男，黑龙江大兴安岭人，博士，清华大学教授、博士生导师，主要研究方向为下一代互联网体系结构、网络测绘等
唐翔宇（1998- ），男，重庆人，清华大学硕士生，主要研究方向为位置网、室内定位等
沈钲晨（1994- ），男，浙江余姚人，清华大学硕士生，主要研究方向为位置隐私等
王会（1977- ），女，河南南阳人，博士，清华大学副教授、博士生导师，主要研究方向为互联网路由、流量工程等
基金资助:
国家重点研发计划基金资助项目(2017YFB0503703)

Nanosecond level time synchronization in datacenter network based on Telemetry architecture

Qianli ZHANG, Chaofan ZHANG, Jilong WANG, Xiangyu TANG, Zhengchen SHEN, Hui WANG

Institute for Network Sciences and Cyberspace, Tsinghua University, Beijing 100084, China

Revised:2021-07-29 Online:2021-10-25 Published:2021-10-01
Supported by:
The National Key Research and Development Program of China(2017YFB0503703)

摘要/Abstract

摘要：

为了解决传统的局部时间同步架构与集中式管理脱节、需要额外交互开销、时间数据较少易受离群值影响等问题，针对数据中心网络的高精准时间同步需求，提出了基于 Telemetry 架构的纳秒级时间同步系统。该系统借助数据中心的背景流量收集时间信息，结合离群因子检测算法对汇报上来的大量时间信息进行分析处理，达成一种集中式的纳秒级自动同步，便于后续全局网络调度。所提系统在可编程交换机上实现并进行了评估，结果显示在多跳之间及不同链路速率下，节点间均可以达到纳秒级的高精准时间同步。

关键词: 高精准时间同步, 纳秒级, 数据中心网络, Telemetry架构

Abstract:

To solve the problems of the traditional local time synchronization architecture being disconnected from centralized management, requiring extra interaction overhead, meanwhile with less time data susceptible to outliers, the nanosecond level time synchronization system was proposed based on Telemetry for the high-precision time synchronization requirements of data center networks.Combined with the outlier factor detection algorithm, timestamp information within the background traffic of the data center was collected, the large amount of time information reported was processed and analyzed, and a centralized automatic synchronization with nanosecond level precision was eventually achieved, which was convenient for subsequent global network scheduling.The proposed system was implemented and evaluated on the programmable switch.Experimental results show that between multiple hops and at different link rates, the nodes can achieve high-precision time synchronization at the nanosecond level.

Key words: high-precision time synchronization, nanosecond level, datacenter network, Telemetry architecture

中图分类号:

TP393

张千里, 张超凡, 王继龙, 唐翔宇, 沈钲晨, 王会. 基于Telemetry架构的数据中心网络纳秒级时间同步[J]. 通信学报, 2021, 42(10): 117-129.

Qianli ZHANG, Chaofan ZHANG, Jilong WANG, Xiangyu TANG, Zhengchen SHEN, Hui WANG. Nanosecond level time synchronization in datacenter network based on Telemetry architecture[J]. Journal on Communications, 2021, 42(10): 117-129.

图/表 18

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

表1

图13

表2

实际数据包发送速率对时间同步效果的影响"

端口额定速率-实际发送速率/(Gbit·s^-1)	用于计算Offset的数据包数量	Offset均值（同步误差）	Offset标准差
10-1	50 000	$38 . 678$	$128 . 622$
10-5	40 800	6.327	$167 . 035$
10-10	17 248	$16 . 037$	$137 . 436$
25-1	50 000	2.315	0.813
25-5	41 072	2.323	0.808
25-10	18 400	2.33	0.793
40-1	50 000	2.288	1.412
40-5	41 202	2.267	4.452
40-10	18 630	2.251	3.951
100-1	50 000	2.343	0.818
100-5	41 498	1.599	1.3
100-10	17 312	2.288	0.798

表2

图14

表3

图15

参考文献 22

[1]	CORBETT J C , HOCHSCHILD P , HSIEH W ,et al. Spanner:Google’s globally distributed database[J]. ACM Transactions on Computer Systems, 2013,31(3): 1-22.
[2]	PERRY J , OUSTERHOUT A , BALAKRISHNAN H ,et al. Fastpass:a centralized “zero-queue” datacenter network[C]// Proceedings of the 2014 ACM Conference on SIGCOMM. New York:ACM Press, 2014: 307-318.
[3]	ZENG H Y , ZHANG S D , YE F ,et al. Libra:divide and conquer to verify forwarding tables in huge networks[C]// 11th USENIX Symposium on Networked Systems Design and Implementation. Berkeley:USENIX Association, 2014: 87-99.
[4]	MILLS D L . Internet time synchronization:the network time protocol[J]. IEEE Transactions on Communications, 1991,39(10): 1482-1493.
[5]	EIDSON J C , FISCHER M , WHITE J . IEEE-1588? standard for a precision clock synchronization protocol for networked measurement and control systems[C]// Proceedings of the 34th Annual Precise Time and Time Interval Systems and Applications Meeting. Piscataway:IEEE Press, 2002: 243-254.
[6]	LEE K S , WANG H , SHRIVASTAV V ,et al. Globally synchronized time via datacenter networks[C]// Proceedings of the 2016 ACM SIGCOMM Conference. New York:ACM Press, 2016: 454-467.
[7]	KANNAN P G , JOSHI R , CHAN M C . Precise time-synchronization in the data-plane using programmable switching ASICs[C]// Proceedings of the 2019 ACM Symposium on SDN Research. New York:ACM Press, 2019: 8-20.
[8]	GENG Y L , LIU S Y , YIN Z ,et al. Exploiting a natural network effect for scalable,fine-grained clock synchronization[C]// Proceedings of the 15th USENIX Conference on Networked Systems Design and Implementation. Berkeley:USENIX Association, 2018: 81-94.
[9]	KIRKPATRICK K . Software-defined networking[J]. Communications of the ACM, 2013,56(9): 16-19.
[10]	TOKUSASHI Y , DANG H T , PEDONE F ,et al. The case for in-network computing on demand[C]// Proceedings of the Fourteenth EuroSys Conference. New York:ACM Press, 2019: 1-16.
[11]	BOSSHART P , DALY D , GIBB G ,et al. P4:programming protocol-independent packet processors[J]. ACM SIGCOMM Computer Communication Review, 2014,44(3): 87-95.
[12]	MURTA C D , JR P R T , MOHAPATRA P . QRPp1-4:characterizing quality of time and topology in a time synchronization network[C]// IEEE Globecom. Piscataway:IEEE Press, 2006: 1-5.
[13]	PáSZTOR A , VEITCH D . PC based precision timing without GPS[C]// Proceedings of the 2002 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems. New York:ACM Press, 2002: 1-10.
[14]	LEE K S , WANG H , WEATHERSPOON H . SoNIC:precise realtime software access and control of wired networks[C]// 10th USENIX Symposium on Networked Systems Design and Implementation. Berkeley:USENIX Association, 2013: 213-225.
[15]	DAVIS M , VILLAIN B , RIDOUX J ,et al. An IEEE-1588 compatible RADclock[C]// 2012 IEEE International Symposium on Precision Clock Synchronization for Measurement,Control and Communication Proceedings. Piscataway:IEEE Press, 2012: 1-6.
[16]	FREEDMAN D A , MARIAN T , LEE J H ,et al. Exact temporal characterization of 10 Gbps optical wide-area network[C]// Proceedings of the 10th Annual Conference on Internet Measurement. New York:ACM Press, 2010: 342-355.
[17]	LI H . IEEE 1588 time synchronization deployment for mobile backhaul in China Mobile[R]. 2014.
[18]	ZARICK R , HAGEN M , BARTO? R , . Transparent Clocks vs.Enterprise Ethernet switches[C]// 2011 IEEE International Symposium on Precision Clock Synchronization for Measurement,Control and Communication. Piscataway:IEEE Press, 2011: 62-68.
[19]	WATT S T , ACHANTA S , ABUBAKARI H ,et al. Understanding and applying precision time protocol[C]// 2015 Saudi Arabia Smart Grid (SASG). Piscataway:IEEE Press, 2015: 1-7.
[20]	BREUNIG M M , KRIEGEL H P , NG R T ,et al. LOF:identifying density-based local outliers[C]// Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2000: 93-104.
[21]	MCKEOWN N , ANDERSON T , BALAKRISHNAN H ,et al. OpenFlow[J]. ACM SIGCOMM Computer Communication Review, 2008,38(2): 69-74.
[22]	BOSSHART P , GIBB G , KIM H S ,et al. Forwarding metamorphosis[J]. ACM SIGCOMM Computer Communication Review, 2013,43(4): 99-110.

交换机资源	原始P4程序stage0	原始P4程序stage1	对照P4程序stage0
8 bit Action Slots	0	0	0
16 bit Action Slots	6.25%	6.25%	6.25%
32 bit Action Slots	3.13%	3.13%	3.13%
Gateway	18.75%	0	6.25%
Hash Bit	4.57%	4.33%	4.57%
SRAM	1.25%	1.25%	1.25%
Stats ALU	0	0	0
TCAM	0	0	0
VLIW Instruction	6.25%	6.25%	3.13%
Exact Match Search Bus	12.50%	6.25%	6.25%

基于Telemetry架构的数据中心网络纳秒级时间同步

Nanosecond level time synchronization in datacenter network based on Telemetry architecture

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 22

相关文章 5

Metrics

推荐阅读 0

端口速率/（Gbit.s^-1）	Offset均值/ns	Offset标准差/ns
25	2.315	0.813
40	2.288	1.412
100	2.343	0.818

[1]	刘文学,陈诗军,葛建,袁洪,龚翠玲. 基于GNSS邻域相似性的5G基站纳秒级时间同步技术研究[J]. 通信学报, 2020, 41(1): 180-190.
[2]	左旭彤,王莫为,崔勇. 低时延网络：架构，关键场景与研究展望[J]. 通信学报, 2019, 40(8): 22-35.
[3]	陈果,张潍丰. ELAB：基于端系统的新型拥塞感知负载均衡机制[J]. 通信学报, 2019, 40(3): 196-205.
[4]	鲍宁海,袁园,刘自谦,匡明. 基于链路生命期的光数据中心网络业务恢复方案[J]. 通信学报, 2018, 39(8): 125-132.
[5]	蔡岳平,王昌平. 软件定义数据中心网络混合路由机制[J]. 通信学报, 2016, 37(4): 44-52.