大数据时代的数据传输网

doi:10.11959/j.issn.2096-0271.2015019

摘要/Abstract

摘要：

大数据时代数据传输需求呈爆炸式增长，如何充分利用数据流的新特征，发挥不同交换方式的优势，是解决未来大数据传输的必由之路。为此，介绍了大数据传输在容量和能耗方面的挑战，并结合数据流的新特征，分析网络技术发展的新机遇。然后，回顾了近几年相关领域的研究进展，并简要介绍了笔者在混合交换和存储转发等方面正在开展的研究工作。

关键词: 大数据网络, 电路/分组混合交换, 存储转发, 数据中心

Abstract:

Explosive growth of traffic demand both inside and between data centers has resulted in a new wave of cutting edge research in the networking community.Inter-and Intra-data center research problems were identified and various solutions aiming to reduce networking cost or increase energy efficiency have been investigated.The growth in traffic demand in the big data era was accompanied by distinct characteristics in flow size distribution and delay tolerance.Data volume was dominated by large flows that are very few in number,and such flows are typically delay tolerant.How these characteristics may be used to tackle the challenge was discussed,followed by our recent work in big data transmission.The BLOC (blocking loss curve) framework aims to provide a framework with which hybrid switching systems combining packet and circuit switching can be studied.The time-shifted multi-layer graph is a generic tool that can be used to analyze the performance of circuit switched networks with bulk storage.

Key words: big data networking, hybrid circuit/packet switching, store and forward, data center

孙卫强, 胡卫生. 大数据时代的数据传输网[J]. 大数据, 2015, 1(2): 56-65.

Weiqiang Sun, Weisheng Hu. Networking Challenges in the Big Data Era[J]. Big Data Research, 2015, 1(2): 56-65.

图/表 5

图1

图2

表1

图3

图4

参考文献 43

[1]	邬贺铨 . 大数据时代的机遇与挑战. 求是, 2013(4)
	Wu H Q . Opportunities and challenges in the era of big data. Qiushi, 2013(4)
[2]	Robinson S , Ferguson R . The storage and transfer challenges of big data. MIT SloanManagement Review, 7 June, 2012
[3]	Greenberg A , Hamilton J , Maltz D A , et al. The cost of a cloud:research problems in data center networks. ACM SIGCOMM Computer Communication Review, 2009,39(1): 68～73
[4]	李国杰程学旗 . 大数据研究:未来科技及经济发展的重大战略领域. 中国科学院院刊, 2012,27(6):647～657
	Li G J , Cheng X Q . Research status and scientific thinking of big data. Bulletin of Chinese Academy of Sciences, 2012,27(6):647～657
[5]	Pollack A . DNA sequencing caught in deluge of data. The New York Times, 12 August, 2011
[6]	Heller B , Seetharaman S , Mahadevan P , et al. ElasticTree:saving energy in data center networks. Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation (NSDI), San Jose,USA, 2010
[7]	Koomey J . Growth in Data Center Electricity Use 2005 to 2010. Oakland: Analytics Press, 2011
[8]	Davis A , Jouppi N P , Mclaren M , et al. The role of photonics in future datacenter networks. Optical Interconnects for Future Data Center Networks, 2013:67～93
[9]	Chen Y , Jain S , Adhikari V K , et al. A first look at inter-data center traffic characteristics via yahoo! datasets. Proceedings of IEEE INFOCOM, Shanghai,China, 2011:1620～1628
[10]	Briscoe B . A fairer,faster internet. IEEE Spectrum, 2008,45(12):42～47
[11]	Laoutaris N , Smaragdakis G , Stanojevic R , et al. Delay-tolerant bulk data transfers on the Internet. IEEE/ACM Transactions on Networking, 2013,21(6):1852～1865
[12]	Laoutaris N , Sirivianos M , Yang X , et al. Inter-datacenter bulk transfers with netstitcher. ACM SIGCOMM Computer Communication Review, 2011,41(4):74～85
[13]	Mahimkar A , Chiu A , Doverspike , et al. Bandwidth on demand for inter-data center communication. Proceedings of the 10th ACM Workshop on Hot Topics in Networks (Hotnets 2011), New York,NY,USA, 2011
[14]	Feng Y , Li B , Li B . Postcard:minimizing costs on inter-datacenter traffic with store-and-forward. Proceedings of the 32nd International Conference on Distributed Computing Systems Workshops (ICDCSW), Macau,China, 2012
[15]	Handley M . Network neutrality and the IETF. Plenary Presentation on IETF, 2009:3～4
[16]	Liou C . Next-generation inter-data center networking,special symposia 2:next generation data centres - paving the way for the Zettabyte era. Proceedings of European Conference and Exhibition on Optical Communication (ECOC), London,UK, 2013
[17]	Ferna?ndez-Palacios J P , Perez L , Rodriguez J , et al. IP off-loading over multi-granular photonic switching technologies. Proceedings of European Conference and Exhibition on Optical Communication (ECOC)2010, Torino,Italy, 2010
[18]	Xin C S , Qiao C M , Ye Y H . A hybrid optical switching approach. Proceedings of Global Telecommunications Conference, San Francisco,USA, 2003:3808～3812
[19]	Lee G M , Wydrowski B , Zukerman M , et al. Performance evaluation of an optical hybrid switching system. Proceedings of Global Telecommunications Conference, San Francisco,USA, 2003:2508～2512
[20]	Vu H L , Zalesky A , Wong E W M , et al. Scalable performance evaluation of a hybrid optical switch. IEEE/OSA Journal of Lightwave Technology, 2005,23(10):2961～2973
[21]	De Leenheer M , Develder C , Vermeir J , et al. Performance analysis of a hybrid optical switch. Proceedings of International Conference on Optical Network Design and Modeling,Vilanova i la Geltru， Spain, 2008
[22]	Veisllari R , Bjornstad S , Bozorgebrahimi K . Integrated packet/circuit hybrid network field-trial, Proceedings of Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference (OFC/NFOEC), Anaheim,CA,USA, 2013
[23]	Wang G , Andersen D G , Kaminsky M , et al. c-Through:part-time optics in data centers. Proceedings of the ACM SIGCOMM, New Delhi,India, 2010
[24]	Singla A , Singh A , Ramachandran K , et al. Proteus:a topology malleable data center network. Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks, Monterey,CA, 2010
[25]	Farrington N , Porter G , Radhakrishnan S , et al. Helios:a hybrid electrical/optical switch architecture for modular data centers. ACM SIGCOMM Computer Communication Review, 2011,41(4):339～350
[26]	Bazzaz H , Tewari M , Wang G , et al. Switching the optical divide:fundamental challenges for hybrid electrical/optical datacenter networks. Proceedings of the 2nd ACM Symposium on Cloud Computing, New York,NY,USA, 2011
[27]	Kachris C , Tomkos I . A survey on optical interconnects for data centers. IEEE Communications Surveys ＆ Tutorials, 2012,14(4):1021～1036
[28]	Sun W Q , Li P Q , Li C . Seamlessly transformable hybrid packet and circuit switching for efficient optical networks. Chinese Optics Letters, 2013,11(1)
[29]	Miyazawa T , Furukawa H , Fujikawa K , et al. Development of an autonomous distributed control system for optical packet and circuit integrated networks. Journal of Optical Communications and Networking, 2012,4(1)
[30]	Vadrevu C S K , Tornatore M , Guok C P , et al. Vertical and horizontal circuit/packet integration techniques for the future optical internet. IEEE Network, 2013,27(1):52～58
[31]	Milton M , Hadi A . Deep packet inspection and bandwidth management:battles over BitTorrent in Canada and the United States. Telecommunications Policy, 2012,36(6):462～475
[32]	Breitgand D , Raz D , Shavitt Y . The traveling miser problem. IEEE/ACM Transactions on Networking, 2006,14(4):711～724
[33]	Chen B B , Primet P . Scheduling deadline-constrained bulk data transfers to minimize network congestion. Proceedings of IEEE/ACM International Symposium on Cluster,Cloud,and Grid Cpmputing, Rio,Brazil, 2007
[34]	Agapi A , Soudan S , Pasin M , et al. Optimizing deadline-driven bulk data transfers in overlay networks. Proceedings of the 18th Internatonal Conference on Computer Communications and Networks, San Francisco,CA,USA, 2009
[35]	Li R , Eryilmaz A . Scheduling for end-to-end deadline-constrained traffic with reliability requirements in multihop networks. IEEE/ACM Transactions on Networking, 2012,20(5):1649～1662
[36]	Miyagi H , Hayashitani M , Ishii D , et al. Advanced wavelength reservation method based on deadline-aware scheduling for lambda grid networks. IEEE/OSA Journal of Lightwave Technology, 2007,25(10):2904～2910
[37]	Andrei D , Tornatore M , Batayneh M , et al. Provisioning of deadline-driven requests with flexible transmission rates in WDM mesh networks. IEEE/ACM Transactions on Networking, 2010,18(2):353～366
[38]	Morell J A . Adaptive resizing of deadline-driven requests for provisioning traffic in elastic optical networks (master dissertation). Miami University, 2013
[39]	Sara R , Agarwal A . SDN approach to largescale global data centers. Proceedings of Open Networking Summit, Santa Clara,California,USA, 2012
[40]	Mckeown N , Anderson T , Balakrishnan H , et al. OpenFlow:enabling innovation in campus networks. ACM SIGCOMM Computer Communication Review, 2008,38(2):69～74
[41]	Das S , Parulkar G , Mckeown N , et al. Packet and circuit network convergence with OpenFlow. Proceedings of Optical Fiber Communication,Collocated National Fiber Optic Engineers Conference, San Diego,CA,USA, 2010
[42]	Yang H , Zhao Y L , Zhang J , et al. Multi-stratum resource integration for OpenFlow-based data center interconnect. IEEE/OSA Journal of Optical Communications and Networking, 2013,5(10):A240～A248
[43]	Liou C . Next-generation inter-data center networking,special symposia 2:next generation data centres - paving the way for the Zettabyte era. Proceedings of European Conference and Exhibition on Optical Communication (ECOC), London,UK, 2013

研究工作	应用范围	结构	策略
光突发交换/光电路交换混合^[19]	边缘节点	并行	长流由光电路交换递送，资源优先分配给光电路交换
光分组交换/光电路交换混合^[29]	骨干节点	集成	部分波长被分组交换和电路交换共享
分组和电路^[30]	骨干节点	集成或并行	将备份资源用于分组或者电路交换
Helios^[25],c-Through^[23]	数据中心	并行	静态资源配置，且不能共享，电路交换优先级高