基于Spark平台的NetFIow流量分析系统

doi:10.3969/j.issn.1000-0801.2014.10.009

Telecommunications Science ›› 2014, Vol. 30 ›› Issue (10): 48-51.doi: 10.3969/j.issn.1000-0801.2014.10.009

• Topic: big data technology and Application • Previous Articles Next Articles

A Large Scale NetFlow Analysis System Based on Spark

Shengyong Ding,Shiwu Min,Yongbing Fan

Guangdong Research Institute of China Telecom Co., Ltd., Guangzhou 510630, China

Online:2014-10-15 Published:2017-06-29

Abstract

Shengyong Ding,Shiwu Min,Yongbing Fan. A Large Scale NetFlow Analysis System Based on Spark[J]. Telecommunications Science, 2014, 30(10): 48-51.

服务器类型	服务器数量	内存	CPU核数	网卡速率	存储
刀片型	8台	8×128 GB	8×16个	1 Gbit／s	8×1 TB

[1]	Xuerong WANG, Zhengzhi TANG, Yinchuan LI, Meiyu QI, Jianbo ZHU, Liang ZHANG. Delay-sensitive traffic intellisense scheduling based on optimal decision tree [J]. Telecommunications Science, 2023, 39(4): 120-132.
[2]	Ming XU,Xue YANG,Jianwu ZHANG. A review of network traffic analysis targeting mobile devices [J]. Telecommunications Science, 2018, 34(4): 98-108.
[3]	Chaoyi BIAN,Shaomin ZHU,Tao ZHOU. Implementation of a big data anonymization system based on Spark [J]. Telecommunications Science, 2018, 34(4): 156-161.
[4]	Zhuohang LI. An improved CLTree algorithm [J]. Telecommunications Science, 2016, 32(8): 124-127.
[5]	Ai Fang,Xiong Xu,Bing Liang,Yuzhong Zhang,Yiping Yang. Comparison of Open-Source Distributed Computing Framework for Big Data [J]. Telecommunications Science, 2015, 31(7): 152-157.
[6]	Xuejiao Chen,Pan Wang,Shidong3 Liu. Key Technology of SSL Encrypted Application Identification Under Imbalance of Application Class [J]. Telecommunications Science, 2015, 31(12): 83-89.
[7]	Ou Liang,Chen Xun,Shen Chen,Huang Xiaoying and Lv Yi. Studies on Traffics Analysis and Prediction Technology for Carrier-Grade IP Network [J]. Telecommunications Science, 2013, 29(7): 24-29.
[8]	Hong Tang. A Large Scale Network Traffic Analysis System Design Based on the MapReduce Platform [J]. Telecommunications Science, 2013, 29(12): 155-157.
[9]	Xiaoming Lu,Weihua Cao,Zhisheng Su,Tong Xu. Analysis of Cloud Computing Solution in IP Backbone NMS of Telecom Operator [J]. Telecommunications Science, 2011, 27(10): 36-40.
[10]	Zhilan Huang,Shengyong Ding,Guoliang Yang,Songfeng Luo. A PCA-based Network Anomaly Detection System Using MapReduce Method [J]. Telecommunications Science, 2010, 26(6): 76-80.
[11]	Guoliang Yang. Analysis of International VoIP Traffic Characterization [J]. Telecommunications Science, 2007, 23(6): 7-15.

A Large Scale NetFlow Analysis System Based on Spark

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 3

Related Articles 11

Metrics

Recommended 0

1	White T . Hadoop: the Definitive Guide. O'Reilly Media Inc, 2012
2	Zaharia M , Chowdhury M , Das T , et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, San Jose, CA, USA, 2012
3	Rossi D , Silvio V . Fine-grained traffic classification with NetFlow data. Proceedings of the 6th International Wireless Communications and Mobile Computing Conference, Shenzhen, China, 2010