从系统角度审视大数据计算

doi:10.11959/j.issn.2096-0271.2015.01.002

大数据 ›› 2015, Vol. 1 ›› Issue (1): 17-26.doi: 10.11959/j.issn.2096-0271.2015.01.002

从系统角度审视大数据计算

郑纬民

清华大学计算机科学与技术系北京 100084

修回日期:2016-05-06 出版日期:2015-05-20 发布日期:2020-09-28
作者简介:郑纬民，男，清华大学教授、博士生导师，中国计算机学会理事长，目前主要从事并行与分布式计算、存储系统的研究工作，主持和参与多项国家“973”计划、“863”计划、国家自然科学基金项目。近年来在IEEE TC/IEEE TPDS/ACM TOS/FAST等本领域顶级期刊与国际会议发表论文40余篇。
基金资助:
国家重点基础研究发展计划（“973”计划）(2014CB340402);国家自然科学基金资助项目(61170008，61272055)

Reviewing Big Data Computation from a System Perspective

Weiming Zheng

Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China

Revised:2016-05-06 Online:2015-05-20 Published:2020-09-28
Supported by:
The National Basic Research Program of China(973 Program)(2014CB340402);The National Natural Science Foundation of China(61170008，61272055)

摘要/Abstract

摘要：

大数据计算是实现大数据“巨大价值”的必要手段，而计算系统是大数据计算的有效载体。试着从系统角度审视大数据计算，透过大数据的体量巨大、速度极快、模态多样、真伪难辨等宏观特征，针对批量计算、流式计算、大图计算等计算形式，分别探讨大数据计算的典型特征，论述了这些特征给大数据计算系统的设计与实现带来的技术挑战，进而梳理了为了应对这些挑战所取得的研究成果，最后从系统角度指出未来大数据计算可能的一些研究方向。

关键词: 大数据计算, 批量计算, 流式计算, 大图计算, 系统实例

Abstract:

Big data computing is a necessary way to acquire the “great value” behind the big data, and a computing system is an effective tool for big data computing. Big data computing from a system perspective was reviewed. Based on the fact that big data has the macro characteristics of huge volume, growing fast, complex structure, and quality disparity, the typical features of big data computing by analyzing batch computing, stream computing, and graph computing respectively, were discussed. These features may bring technical challenges to the design and implementation of big data computing system. The related works for overcoming these challenges were further categoried. In the end, some prospective research directions of big data computing from the system perspective were listed.

Key words: big data computing, batch computing, stream computing, graph computing, system instance

郑纬民. 从系统角度审视大数据计算[J]. 大数据, 2015, 1(1): 17-26.

Weiming Zheng. Reviewing Big Data Computation from a System Perspective[J]. Big Data Research, 2015, 1(1): 17-26.

图/表 6

参考文献 21

[1]	Chen C L , Zhang C Y . Data-intensive applications, challenges, techniques and technologies: a survey on big data. Information Sciences, 2014(275): 314-347.
[2]	Chang R M , Kauffman R J , Kwon Y . Understanding the paradigm shift to computational social science in the presence of big data. Decision Support Systems, 2014(63): 67-80.
[3]	Kambatla K , Kollias G , Kumar V et al . Trends in big data analytics. Journal of Parallel and Distributed Computing, 2014(74): 2561-2573.
[4]	李国杰, 程学旗 . 大数据研究: 未来科技及经济社会发展的重大战略领域——大数据的研究现状与科学思考. 中国科学院院刊, 201227(6): 647～657
	Li G J , Cheng X Q . Big data research: the major strategic areas of technology and economic development——research status and scientific thinking of big data. Bulletin of the Chinese Academy of Sciences, 2012,27(6): 647～657.
[5]	孙大为, 张广艳, 郑纬民 . 大数据流式计算：关键技术及系统实例. 软件学报, 201425(4): 839～862
	Sun D W , Zhang G Y , Zheng W M . Big data stream computing: technologies and instances Journal of Software, 2014,25(4): 839～862
[6]	程学旗, 靳小龙, 王元卓等. 大数据系统和分析技术综述. 软件学报, 201425(9): 1889～1908
	Cheng X Q , Jin X L , Wang Y Z et al. Survey on big data system and analytic technology. Journal of Software, 201425(9): 1889～1908
[7]	王元卓, 靳小龙, 程学旗 . 网络大数据：现状与展望. 计算机学报, 201336(6): 1125～1138.
	Wang Y Z , Jin X L , Cheng X Q . Network big data: present and future. Chinese Journal of Computers, 201336(6): 1125-1138
[8]	李学龙, 龚海刚 . 大数据系统综述. 中国科学:信息科学 201545(1): 1～44
	Li X L , Gong H G . Survey on big data system. Scientia Sinica Informationis, 201545(1): 1～44
[9]	Dobre C , Xhafa F . Intelligent services for big data science. Future Generation Computer Systems, 2014(37): 267～281.
[10]	Aisling O D , Jurate D , Roy D S . Hadoop and cloud computing in genomics. Journal of Biomedical Informatics, 2013(46): 774～781.
[11]	Hadoop .
[12]	Zaharia M , Das T , Li H et al. Discretized streams: fault-tolerant streaming computation at scale. Proceedings of the SOSP 2013, Pennsylvania, USA, 2013
[13]	Spark. .
[14]	Cugola G , Margara A , Li H . Processing flows of information: from data stream to complex event processing. ACM Computing Surveys, 2011,44(3): 51～62.
[15]	Zhang Z , Gu Y , Ye F . et al. A hybrid approach to high availability in stream processing systems. Proceedings of the 30th IEEE International Conference on Distributed Computing Systems, Genova, Italy, 2010,135-148.
[16]	Liu X F , Lftikhar N , Xie X . Survey of real-time processing systems for big data. Proceedings of the 30th IEEE International Conference on Distributed Computing Systems, Genova, Italy, Jun 2014,356-361.
[17]	Storm.
[18]	Chauhan J , Chowdhury S A , Makaroff D . Performance evaluation of Yahoo! S4: a first look. Proceedings of 7th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing， Victoria, BC, Canada 2012,58-65.
[19]	Chatziantoniou D , Pramatari K , Sotiropoulos Y . Supporting real-time supply chain decisions based on RFID data streams. Journal of Systems and Software, 2011,84(4): 700-710.
[20]	GraphLab. .
[21]	Furedi Z , Kostochka A , Kumbhat M . Choosability with separation of complete multipartite graphs and hypergraphs. Journal of Graph Theory, 2014,76(2): 129-137.

从系统角度审视大数据计算

Reviewing Big Data Computation from a System Perspective

在线阅读

PDF下载

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 21

相关文章 2

Metrics

推荐阅读 0

[1]	吴城文, 张广艳, 郑纬民. 从系统角度审视大图计算[J]. 大数据, 2015, 1(3): 48-61.
[2]	孙大为. 大数据流式计算：应用特征和技术挑战[J]. 大数据, 2015, 1(3): 99-105.