大数据技术的应用现状与展望

doi:10.11959/j.issn.1000-0801.2015189

Abstract

Abstract:

The research includes computer science，information science，management science，psychology，etc.Four key technologies in big data research，data collection，preprocessing and storage，data analysis and mining，data privacy protection，data center system structure were discussed and the hot research topics from domestic and overseas research groups were demonstrated in order to have comprehensive and systematic understanding of big data.The application status of big data was introduced from three perspectives，application platforms，relevant products and standardization.Meanwhile，description of current business in big data and other field was presented.Eventually，the opportunities and challenges in big data research were revealed through big data analysis strategies，privacy protection and data quality，which provide the guideline for future research.

Key words: big data, data analysis, data mining, unstructured data, privacy protection, parallel computing, social network data

Jianxin Liao. Big Data Technology：Current Applications and Prospects[J]. Telecommunications Science, 2015, 31(7): 1-12.

Figures/Tables 5

References 44

1	Gantz J ， Reinsel D . Extracting Value from Chaos. IDC iView Report， 2011
2	Schnberger V M ， Cukier K . 大数据时代：生活、工作与思维的大变革. 盛杨燕，周涛译. 杭州：浙江人民出版社， 2013 Schnberger V M ， Cukier K . Big Data：A Revolution that Will Transform How We Live，Work，and Think. Translated by Sheng Y Y，Zhou T. Hangzhou： Zhejiang People’s Publishing House， 2013
3	Team R . Big Data Now：Current Perspectives from O’Reilly Radar. Sebastopol：O’Reilly Media， 2011
4	Grobelnik M . Big data tutorial. ， 2012
5	张引，邹传伟，廖小飞 . 大数据应用的现状与展望. 计算机研究与发展， 2013，50（s2） Zhang Y ， Chen M ， Liao X F . Big data applications：a survey. Journal of Computer Research and Development， 2013，50（s2）
6	Binzenh?fer A ， Tutschku K ， Graben B A D ，et al. A P2P-based framework for distributed network management. Lecture Notes in Computer Science， 2006（3883）： 198～210
7	Tutschku K ， Chevul S ， Binzenhfer A ， Schmid M ，et al. A self-organizing concept for distributed end-to-end quality monitoring. University of Wurzburg Institute，Wurzburg，Germany， 2006
8	李强，王宏，王乐春 . 基于P2P 的分布式网络管理模型研究. 计算机工程， 2006，32（13）： 150～152. Li Q ， Wang H ， Wang L C . Research of P2P based distributed network management. Computer Engineering， 2006，32（13）： 150～152
9	Karagiannis T ， Papagiannaki K ， Faloutsos M . Blinc：multilevel traffic classificaion in the dark. Proceedings of the 2005 Conference on Applications，Technologies，Architectures，and Protocols for Computer Communications， Philadelphia，Pennsylvania，USA， 2005
10	Karagiannis T ， Roido A ， Aloutsos M ，et al. Transport layer identification of P2P traffic.In： Proceedings of the 2004 ACM SIGCOMM Internet Measurement Conference， Taormina，Italy， 2004
11	鲍静，范生万 . 基于数据挖掘的图书数据预处理. 大学图书情报学刊， 2008，26（2）： 31～33. Bao J ， Fan S W . Pre-processing library data based on data milling. Journal of Academic Library and Information Science， 2008，26（2）： 31～33
12	刘云霞 . 数据归约的统计方法研究及应用（博士学位论文）. 厦门大学， 2007 Liu Y X . Research and application of the statistical method of data reduction（doctor dissertation）. Xiamen University， 2007
13	刘鹏 . 云计算技术基础. 北京：电子工业出版社， 2011 Liu P . Fundamental Cloud Computing Technology. Beijing： Publishing House of Electronics Industry， 2011
14	王庆波，金涬，何乐 . 虚拟化与云计算. 北京：电子工业出版社， 2010 Wang Q B ， Jin X ， He L . Virtualization and Cloud Computing. Beijing： Publishing House of Electronics Industry， 2010
15	王鹏 . 云计算的关键技术与应用实例. 北京：人民邮电出版社， 2010 Wang P . Key Technologies and Applications of Cloud Computing. Beijing： Posts and Telecom Press， 2010
16	Agrawal D ， Bernstein P ， Bertino E ，et al. Challenges and opportunities with big data. Challenges and Opportunities with Big Data-ResearcGate， 2012，6（12）： 2032～2033
17	Luo D ， Ding C ， Huang H . Parallelization with multiplicative algorithms for big data mining. Proceedings of IEEE 12th International Conference on Data Mining， Brussels，Belgium， 2012： 489～498
18	Gao F ， Abd-Almageed W ， Hefeeda M . Distributed approximate spectral clustering for large-scale datasets. Proceedings of the 21st International ACM Symposium on High-Performance Parallel and Distributed Computing， Delft，the Netherlands， 2012： 223～234
19	Shim K . MapReduce algorithms for big data analysis，and storage of big data. Proceedings of the VLDB Endowment， Istanbul，Turkey， 2012： 2016～2017
20	Abadi D J ， Ahmad Y ， Balazinska M ，et al. The design of the borealis stream processing engine. Proceedings of 2nd Biennial Conference on Innovative Data Systems Research（CIDR’05）， Asilomar，CA，USA， 2005
21	Motwani R ， Widom J ， Arasu A ，et al. Query processing，resource management，and approximation in a data stream management system. Proceedings of the 2003 CIDR Conference， Asilomar，CA，USA， 2003
22	Han J W ， Kamber M ， Pei J ，et al. 数据挖掘：概念与技术. 范明，孟小峰译. 北京：机械工业出版社， 2005 Han J W ， Kamber M ， Pei J ，et al. Data Mining：Concepts and Techniques.Translated by Fan M，Meng X F. Beijing： China Machine Press， 2005
23	Al-Fhres M ， Loukissas A ， Vahdat A . A scalable，commodity data center network architecture. Proceedings of ACM SIGCOMM， Seattle，WA，USA， 2008
24	Mitchell B ， Sheppard J . Deep structure learning：beyond connectionist approaches. Proceedings of the 11th International Conference on Machine Learning and Applications（ICMLA）， Boca Raton，Florida，USA， 2012： 162～167
25	Hinton G ， Osindero S ， Teh Y W . A fast learning algorithm for deep belief nets. Neural Computation， 2006，18（7）： 1527～1554
26	FuKusHIMA K . Neocognitron：a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybemetics， 1980，36（4）： 193～202
27	Ranzato M ， Susskind J ， Mnih V ，et al. On deep generative models with applications to recognition. Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）， Colorado Springs，CO，USA， 2011： 2857～2864
28	Guo C ， Wu H ， Tan K ，et al. DCell：a scalable and fault-tolerant network structure for data center. Proceedings of ACM SIGCOMM， Seattle，WA，USA， 1998
29	Li D ， Guo C ， Wu H ，et al. FiConn：using backup port for server interconnection in data centers. Proceedings of IEEE INFOCOM， Rio de Janeiro，Brazil， 2009
30	胡雄伟，张宝林，李抵飞 . 大数据研究与应用综述. 标准科学， 2013（10）： 18～21 Hu X W ， Zhang B L ， Li D F . Overview of big data research and application. Standard Science， 2013（10）： 18～21.
31	方巍，文学志，潘吴斌等. 云计算：概念、技术及应用研究综述. 南京信息工程大学学报：自然科学版， 2012,4（4）： 351～361 Fang W ， Wen X Z ， Pan W B ，et al. Cloud computing：survey on concept，technology and application. JJournal of Nanjing University of Information Science ＆ Technology， 2012，4（4）： 351～361
32	Chandarana P ， Vijayalakshmi M . Big data analytics frameworks. Proceedings of 2014 International Conference on Circuits，Systems，Communication and Information Technology Applications （CSCITA）， Mumbai，India， 2014： 430～434
33	中国电子技术标准化研究院. 大数据标准化白皮书， 2014China Electronics Standardization Institute. Big Data Standardization White Paper， 2014
34	宫夏屹，李伯虎，柴旭东等. 大数据平台技术综述. 系统仿真学报， 2014，26（3）： 489～496 Gong X Y ， Li B H ， Chai X D ，et al. Survey on big data platform technology. Journal of System Simulation， 2014，26（3）： 489～496
35	黄哲学，陈小军，李俊杰等. 面向服务的大数据分析平台解决方案. 科技促进发展， 2014，10（1）： 52～59 Huang Z X ， Chen X J ， Li J J ，et al. Service oriented big data analytics platform. Science ＆ Technology for Development， 2014，10（1）： 52～59
36	袁玮 . 云计算在电信行业经营分析系统中对大数据的处理探析. 硅谷， 2014（6） Yuan W . Processing analysis for big data from cloud computing in telecom industry management analysis system. Silicon Valley， 2014（6）
37	Demchenko Y ， De Laat C ， Membrey P . Defining architecture components of the big data ecosystem. Proceedings of 2014 International Conference on Collaboration Technologies and Systems（CTS）， Minneapolis，MN，USA， 2014： 104～112
38	Zeng D ， Lusch R . Big data analytics：perspective shifting from transactions to ecosystems. IEEE Intelligent Systems， 2013，28（2）： 2～5
39	Agarwal S ， Mozafari B ， Panda A ，et al. BlinkDB：queries with bounded errors and bounded response times on very large data. Proceedings of the 8th ACM European Conference on Computer Systems， Prague，Czech Republic， 2013： 29～42
40	孟小峰 . 位置大数据隐私保护研究综述. 软件学报， 2014，25（4）： 693～712 Meng X F . Location privacy preservation in big data era：a survey. Journal of Software， 2014，25（4）： 693～712
41	Cheng Y ， Park J ， Sandhu R . Preserving user privacy from third-party applications in online social networks. Proceedings of the 22nd International Conference on World Wide Web Companion， New York，USA， 2013： 723～728
42	Ghosh A ， Roughgarden T ， Sundararajan M . Universally utility-maximizing privacy mechanisms. Proceedings of the 22nd International Conference on World Wide Web Companion， Bethesda，Maryland，USA， 2009： 351～360
43	李默涵，李建中，高宏 . 数据时效性判定问题的求解算法. 计算机学报， 2012，35（11）： 2348～2360 Li M H ， Li J Z ， Gao H . Evaluation of data currency. Chinese Journal of Computers， 2012，35（11）： 2348～2360
44	刘波，耿寅融 . 数据质量检测规则挖掘方法. 模式识别与人工智能， 2012，25（5）： 835～844 Liu B ， Geng Y R . Mining method for data quality detection rules. Pattern Recognition and Artificial Intelligence， 2012，25（5）： 835～844

Metrics

Recommended 0

No Suggested Reading articles found!

[1]	Lu HAN, Weiyu CHEN, Fei ZHANG, Jianfeng HE, Huaizhen SU. Big data classification method of non relational distributed submission information under differentiated requirements [J]. Telecommunications Science, 2023, 39(6): 114-121.
[2]	Xue HAN. Damage toughness assessment method of power backbone communication network based on power big data [J]. Telecommunications Science, 2023, 39(5): 136-143.
[3]	Yudi SUN. A prediction model of massive 5G network users’ revisit behavior based on telecom big data [J]. Telecommunications Science, 2023, 39(2): 157-162.
[4]	Aihua LI, Xiaobo WU, Chao CHEN, Bin WEI, Yuanyuan SHI. Big data intelligent analysis technology for 5G network [J]. Telecommunications Science, 2022, 38(8): 129-139.
[5]	Gengsheng HUANG, Yuhong HUANG, Manxue GUO, Jianping ZHENG, Xin GE. Architecture of a new infrastructure for data service in DT era [J]. Telecommunications Science, 2022, 38(7): 138-145.
[6]	Panpan LI, Zhengxia XIE, Guangxue YUE, Xin LIU. Research progress and trends of deep learning based wireless communication receiving method [J]. Telecommunications Science, 2022, 38(2): 1-17.
[7]	Haibo ZHAO, Zhijun XIANG, Linsong XIAO. A big data framework for short-term power load forecasting using heterogenous data [J]. Telecommunications Science, 2022, 38(12): 103-111.
[8]	Jian HU, Lei SUN, Yiyan YIN, Xiaokang YANG, Jingui BAI, Yejiang ZHANG. Research on 4G/5G voice quality optimization based on data mining and geographic visualization [J]. Telecommunications Science, 2022, 38(10): 131-139.
[9]	Zhiyong LIU, Zhongjiang HE, Yilong RUAN, Junfeng SHAN, Chao ZHANG. Big data security features and operation practices [J]. Telecommunications Science, 2021, 37(5): 160-169.
[10]	Chi ZHANG, Ye LU, Yuping LUO, Xiaokai SUN, Hanke ZHU. A novel approach for face privacy protection based on surveillance video in complex scene [J]. Telecommunications Science, 2021, 37(1): 94-101.
[11]	Xiaojun LIU,Juan WU,Xiaoqing XU. Big data architecture analysis and data security fusion technology [J]. Telecommunications Science, 2020, 36(7): 146-155.
[12]	Shaoliang PENG,Liang BAI,Li WANG,Minxia CHENG,Shulin WANG. Trusted edge computing for smart healthcare [J]. Telecommunications Science, 2020, 36(6): 56-63.
[13]	Yujiao LIU,Kunhuang SONG,Xiang WANG. Analysis of economic prosperity index based on big data of electricity [J]. Telecommunications Science, 2020, 36(6): 166-171.
[14]	Zishan LIU,Qiang CHENG,Bo LV. A survey on key technologies of privacy protection for machine learning [J]. Telecommunications Science, 2020, 36(11): 18-27.
[15]	Weihua CHENG,Xiaorong HE. Quality analysis method of Web browsing based on big data [J]. Telecommunications Science, 2020, 36(11): 174-181.

Big Data Technology：Current Applications and Prospects

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 44

Related Articles 15

Metrics

Recommended 0