基于Hadoop的电信BSS大数据平台建设研究

doi:10.3969/j.issn.1000-0801.2013.03.008

Abstract

Abstract:

Te1ecom operators are transforming their business mode1 from“voice+SMS+VAS”to“voice+app1ication+traffic”,and they are choosing the mobi1e traffic operation as their key deve1opment strategy.In this scenario,big data is becoming the next frontier for the te1ecom IT support system.A distributed architecture based on Hadoop to provide big data management and services for traffic operation was described,and with the experimenta1 data,demonstration of the outstanding performance of the p1atform was introduced.

Key words: big data, p1atform, Hadoop, Hbase, high avai1abi1ity

Na Chen,Xinyi Xu,Hongbing Song,Yi He. Research on Big Data System of Telecom Based on the Hadoop[J]. Telecommunications Science, 2013, 29(3): 36-40.

业务类型	文件编码方式	原始记录字段数（条）	需入库的记录字段数（条）	需入库记录数（条）
HTTP业务	ASCII	65	14	25亿
WAP业务	ASCII	40	14	25亿

评估点	评估指标
50亿条数据的入库性能	入库总耗时（LTC）、每秒入库记录数（RNPS）
50亿条数据的查询性能	平均查询响应时长（ATRT）、平均事务处理能力（TPS）
高可用性验证	是否有单点问题、RTO与RPO
初次投资成本	成本

	LTC	RNPS（万条/s）
传统IT架构	11时46分	11.8
本平台	5时20秒	28.1

查询方法	虚拟用户数（个）	传统IT架构ATRT（s）	传统IT架构TPS	本平台ATRT（s）	本平台TPS
查50亿条	300	0.457	1 829	0.038	3 769
边入边查50亿条	300	0.461	1 684	0.045	3 132
边删边查50亿条	300	0.459	1 775	0.039	3 669

验证点	是否有单点问题	RTO	RPO
停查询应用的非主节点	否	0	0
停查询应用的主节点	否	3s	0
停Hadoop的非主节点	否	0	0
停Hadoop的主节点	否	0	0

Research on Big Data System of Telecom Based on the Hadoop

RichHTML

PDF下载

Knowledge

Cited

Abstract

Cite this article

share this article

Figures/Tables 12

References 8

Related Articles 15

Metrics

Recommended 0

成本项目	数量	配置	总价（元）
DL380 Gen8服务器	6	E5-2630×1（12核）16 GB内存、2.3 TB存储空间	48万
操作系统	6	Redhat	6万
系统软件	1	Hadoop+Hbase+Zookeeper	0
App中间件	1	Tomcat	0
合计			54万

1	Borthakur D . Hadoop distributed file system. ， 2007
2	White T . Hadoopthe Definitive Guide， United States of America:O’Reilly， 2009
3	Owen O’Malley . Programming with Hadoop’s Map Reduce. Apache Con EU， 2008
4	Apache . We1cometo Hadoop. ， 2007
5	Borthakur D . The Hadoop distributed fi1e system：architectureanddesign. ， 2011
6	David ECuller ， Jaswinder Pal ， Singh Anoop 等著.李晓明译. 并行计算机体系结构：硬件/软件结合的设计与分析. （第2版）北京：机械工业出版社， 2003
7	曹羽中 . 用Hadoop进行分布式并行编程（第1部分）. ， 2008
8	朱珠 . 基于Hadoop的海量数据处理模型研究和应用. 北京邮电大学硕士学位论文， 2008

[1]	Lu HAN, Weiyu CHEN, Fei ZHANG, Jianfeng HE, Huaizhen SU. Big data classification method of non relational distributed submission information under differentiated requirements [J]. Telecommunications Science, 2023, 39(6): 114-121.
[2]	Xue HAN. Damage toughness assessment method of power backbone communication network based on power big data [J]. Telecommunications Science, 2023, 39(5): 136-143.
[3]	Yudi SUN. A prediction model of massive 5G network users’ revisit behavior based on telecom big data [J]. Telecommunications Science, 2023, 39(2): 157-162.
[4]	Aihua LI, Xiaobo WU, Chao CHEN, Bin WEI, Yuanyuan SHI. Big data intelligent analysis technology for 5G network [J]. Telecommunications Science, 2022, 38(8): 129-139.
[5]	Gengsheng HUANG, Yuhong HUANG, Manxue GUO, Jianping ZHENG, Xin GE. Architecture of a new infrastructure for data service in DT era [J]. Telecommunications Science, 2022, 38(7): 138-145.
[6]	Panpan LI, Zhengxia XIE, Guangxue YUE, Xin LIU. Research progress and trends of deep learning based wireless communication receiving method [J]. Telecommunications Science, 2022, 38(2): 1-17.
[7]	Haibo ZHAO, Zhijun XIANG, Linsong XIAO. A big data framework for short-term power load forecasting using heterogenous data [J]. Telecommunications Science, 2022, 38(12): 103-111.
[8]	Zhiyong LIU, Zhongjiang HE, Yilong RUAN, Junfeng SHAN, Chao ZHANG. Big data security features and operation practices [J]. Telecommunications Science, 2021, 37(5): 160-169.
[9]	Xiaojun LIU,Juan WU,Xiaoqing XU. Big data architecture analysis and data security fusion technology [J]. Telecommunications Science, 2020, 36(7): 146-155.
[10]	Yujiao LIU,Kunhuang SONG,Xiang WANG. Analysis of economic prosperity index based on big data of electricity [J]. Telecommunications Science, 2020, 36(6): 166-171.
[11]	Weihua CHENG,Xiaorong HE. Quality analysis method of Web browsing based on big data [J]. Telecommunications Science, 2020, 36(11): 174-181.
[12]	Xiaomin TAN,Ai FANG,Bing LIANG,Haojie YANG. Locating causes of abnormality of EPG experience based on decision tree [J]. Telecommunications Science, 2019, 35(8): 158-164.
[13]	Zelin LU,Qiangzhi LI. Evolutionary logic and operational mechanism of E-commerce platform [J]. Telecommunications Science, 2019, 35(7): 152-158.
[14]	Ming LIAO,Ming CHEN,Ji ZHOU,Xiaohua XIANG,Fang LI,Yefen JIAO. DNS log analysis system based on big data fusion algorithm [J]. Telecommunications Science, 2019, 35(5): 129-139.
[15]	Yuanfeng GONG,Yi HUANG. Optimization method of wireless indoor coverage based on big data analysis [J]. Telecommunications Science, 2019, 35(5): 149-154.

成本项目	数量	配置	总价（元）
HP rx8640服务器	2	16CPU/128 GB内存	200万
存储	1	12 TB（HP EVA8400）	10万
数据库软件	1	Orac1e10g	80万
App中间件	1	Tomcat	0
合计			290万