面向大数据的数据服务中心设计与应用研究

doi:10.3969/j.issn.1000-436x.2013.Z2.032

Abstract

Abstract:

Big data management gives challenges to data manage and service platform based on traditional relational database management system,and research on big data management and service platform supporting both structured and unstructured data has become a very urgent and important task.4 key technologies for big data management service platform was summarized,and the configurable generalization table model was adopt.The data service center system prototype BIG-DMS based on Hadoop platform and the application scheme of data integration services platform based on BIG-DMS were also designed to support both structured and unstructured data storage,management and access services.

Key words: big data, MapReduce, data model,, data service center

Yun-feng LIU,Li LI,Su-mei WANG,Qian-yi WANG,Xu YANG,Rong-bin OUYANG,Xin-zheng LONG. Design and application research on data service platform for big data[J]. Journal on Communications, 2013, 34(Z2): 170-174.

Figures/Tables 2

References 10

[1]	Big data[EB/OL]. , 2011.
[2]	ZHOU X F , LU J H , LI C P ,et al. The challenges of big data from the perspective of data management[J]. Communications of the China Computer Federation, 2012,8(9): 16-21.
[3]	北京大学计算中心北京大学电子校务信息门户项目方案[R]. 2006.Computer Center of Peking University Peking University Electronic Affairs Information Portal Project[R]. 2006.
[4]	王倩宜, 李丽, 欧阳荣彬 ,等. 数据综合服务管理平台主体方案探讨[J]. 实验技术与管理, 2011,28(5): 4-6. WANG Q Y , LI L , OUYANG R B ,et al. Discussion on infrastructure proposal of data integration and service platform[J]. Experimental Technology and Management, 2011,28(5): 4-6.
[5]	龙新征, 欧阳荣彬, 王倩宜等. 基于SSI模型的综合查询系统的设计与实现[J]. 武汉大学学报(理学版), 2012,58(21): 242-246. LONG X Z , OUYANG R B , WANG Q Y ,et al. Design and implementationof a comprehensive query system based on SSI model[J]. Journey of Wuhan University (Natural Science Edition), 2012,58(21): 242-246.
[6]	CHENG X , MENG B P , CHEN Y X ,et al. Dynamic table:a layered and configurable storage structure in the cloud[A]. WAIM Workshops[C]. 2012. 204-215.
[7]	Hadoop[EB/OL]. . 2011.
[8]	DEAN J , GHEMAWAT S . MapReduce:simplified data processing on large clusters[A]. Proc of the OSDI[C]. California,USA, 2004. 137-150.
[9]	OUSTERHOUT J , AGRAWAL P , ERICKSON D ,et al. The case for ramcloud[J]. Communications of the ACM, 2011,54(7): 121-130.
[10]	ZAHARIA M , CHOWDHURY M , DAS T ,et al. Resilient distributed datasets:a fault-tolerant abstraction for in-memory cluster computing[A]. Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation[C]. 2012. 1-14.

Metrics

Recommended 0

No Suggested Reading articles found!

[1]	Wei JIN, Fenghua LI, Mingjie YU, Yunchuan GUO, Ziyan ZHOU, Liang FANG. HDFS-oriented cryptographic key resource control mechanism [J]. Journal on Communications, 2022, 43(9): 27-41.
[2]	Yimin MAO, Qianhu DENG, Zhigang CHEN. Parallel association rules incremental mining algorithm based on information entropy and genetic algorithm [J]. Journal on Communications, 2021, 42(5): 122-136.
[3]	Liang YUAN, Xiao YU, Enjie DING, Xiaohu ZHAO, Shimin FENG, Da ZHANG, Tongyu LIU, Weidong WANG, Yanqiu HUANG. Research on key technologies of human-machine-environment states perception in mine Internet of things [J]. Journal on Communications, 2020, 41(2): 1-12.
[4]	. Dual-architecture Internet supporting intelligent governance of cyber content [J]. Journal on Communications, 2019, 40(9): 1-14.
[5]	Yonglin PU,Jiong YU,Liang LU,Ziyang LI,Chen BIAN,Bin LIAO. Energy-efficient strategy for data migration and merging in Storm [J]. Journal on Communications, 2019, 40(12): 68-85.
[6]	Yu FU, Yihan YU, Xiaoping WU. Differential privacy protection technology and its application in big data environment [J]. Journal on Communications, 2019, 40(10): 157-168.
[7]	Yuanzhao GAO,Binglong LI,Xingyuan CHEN. Stochastic algorithm for HDFS data theft detection based on MapReduce [J]. Journal on Communications, 2018, 39(10): 11-21.
[8]	Yonglin PU,Jiong YU,Liang LU,Chen BIAN,Bin LIAO,Ziyang LI. Energy-efficient strategy for work node by DRAM voltage regulation in storm [J]. Journal on Communications, 2018, 39(10): 97-117.
[9]	Yihan YU,Yu FU,Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework [J]. Journal on Communications, 2018, 39(1): 70-77.
[10]	Kun-fang ZHANG,Ming-ming LU,Lin ZHENG. Big data based metro crowd delivery system [J]. Journal on Communications, 2017, 38(Z2): 99-112.
[11]	Le-tian SHA,Fu XIAO,Wei CHEN,Jing SUN,Ru-chuan WANG. Sensitive information leakage awareness method for big data platform based on multi-attributes decision-making and taint tracking [J]. Journal on Communications, 2017, 38(7): 56-69.
[12]	Jun-xin SHEN,Ying-qian CHEN. Research on network analysis method for development ability of big data industry in underdeveloped area [J]. Journal on Communications, 2017, 38(12): 153-159.
[13]	Lin ZHANG,Yan LIU,Ru-chuan WANG. Location publishing technology based on differential privacy-preserving for big data services [J]. Journal on Communications, 2016, 37(9): 46-54.
[14]	Shao-feng GENG,Yong-heng WANG,Ren-fa LI,Jia ZHANG. Research of proactive complex event processing method [J]. Journal on Communications, 2016, 37(9): 111-120.
[15]	Fei ZHU,Zhi-peng XU,Quan LIU,Yu-chen FU,Hui WANG. Online hierarchical reinforcement learning based on interrupting Option [J]. Journal on Communications, 2016, 37(6): 65-74.

Design and application research on data service platform for big data

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 2

References 10

Related Articles 15

Metrics

Recommended 0