复本存储机制的效率研究

doi:10.3969/j.issn.1000-436x.2013.07.013

摘要/Abstract

摘要：

建立独立于具体算法的复本存储模型，识别影响效率的关键参数和无关参数；分析系统规模增长对效率的影响，通过实验得出空间效率与I/O效率的乘积同系统节点数量近似成反比这一结论，并从理论上解释结论背后的成因。研究结果可用于指导复本存储机制的工程应用，为空间效率和I/O效率之间的权衡以及大规模存储系统I/O性能的预测提供理论依据。

关键词: 存储系统, 复本存储, 负载均衡, I/O效率, 空间效率

Abstract:

An algorithm-independent description model was established for the ism, and both efficiency-related and efficiency-independent parameters were identified. Besides,the impact of system scale on efficiency was investigated, and an important observation was obtained that the product of the two utilization ratios (i.e., the utilization ratio of disk space and that of I/O bandwidth) is approximately inversely proportional to the number of nodes contained in the system. The observation was verified through experimental resu reason behind was revealed through theoretical analy-sis. The study provides guidelines for the application of replication-based storage mechanism in engineering, and lays a theoretical foundation for the trade-off between the two utilization ratios and for the prediction of I/O performance in large-scale storage systems.

Key words: storagesystem, replication-basedstorage, oadbalance, I/Oefficiency, spaceefficiency

罗香玉,汪芸,陈笑梅,袁飞飞,李聪. 复本存储机制的效率研究[J]. 通信学报, 2013, 34(7): 111-123.

Xiang-yu LUO,Yun WANG,Xiao-mei CHEN,Fei-fei YUAN,Con LI. Research on the efficiency of replication-based storage mechanism[J]. Journal on Communications, 2013, 34(7): 111-123.

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

参考文献 26

[1]	聂雪军．内容感知存储系统中信息生命周期管理关键技术研究[D]. 武汉：华中科技大学， 2011. NIE X J . Research on Key Technologies of Information LifecycManagement in Content Aware Storage System[D]. Wuhan: Huazhong University of Science and Technology, 2011.
[2]	KIM J , CHOU J , ROTEM D . Energy proportionality and performance in data parallel computing clusters[A]. Proceedings of Scientific and Statistic-al Database Management Conference[C]. Portland, USA, 2011. 414-431.
[3]	陆承涛 . 存储系统性能管理问题的研究[D]．武汉: 华中科技大学, 2010. LU C T . Performance Management in Storage Systems[D]. uhan: Huazhong University of Science and Technology, 2010.
[4]	AGUILERA M K , JANAKIRAMAN R , XU L . Using erasure codes efficiently for storage in a distributed system[A]. Pr ceedings of Interna-tional Conference on Dependable Systems and Networks[C]. Yokohama, Japan, 2005. 336-345.
[5]	MACCORMICK J , MURPHY N , RAMASUBRAMANIAN V , et a l . Kinesis: a new approach to replica placement in distributed storage systems[J]. ACM Transactions on Storage, 2009,4(4):1-28.
[6]	BRESLAU L , CAO P , FAN L , et al . Web caching and zip-like distri-butions: evidence and implications[A]. Proceedings of INFOCOM[C]. New York, USA, 1999. 126-134.
[7]	TAN W , FU Y , CHERKASOVA L , et al . MediSyn:a synthetic stream-ing media service workload generator[A]. Proceedings o the 13th In-ternational Workshop on Network and Operating Systems Support for Digital Audio and Video[C]. Monterey, USA, 2003. 12-21.
[8]	DABEK F , KAASHOEK M F , KARGER D , et al . Wide-area coopera-tive storage with CFS[A]. Proceedings of the 18th ACM mposium on Operating Systems Principles[C]. Banff, Canada, 2001. 202-215.
[9]	VAN RENESSE R , SCHNEIDER F B . Chain replication for support-ing high throughput and availability[A]. Proceedings of the Sympo-sium on Operating Systems Design and Implementation[C]. San Fran-cisco, USA, 2004. 91-104.
[10]	XIE J , CAI B . A decentralized storage cluster with hig liability and flexibility[A]. Proceedings of the 14th Euromicro International Con-ference on Parallel, Distributed and Network-Based Processing[C]. Montbeliard-Sochaux, France, 2006. 116-123.
[11]	WANG W , ZHAO Y . A novel network storage scheme: intelligent network disk storage cluster[A]. Proceedings of the 5th International Conference on Networking, Sensing and Control[C]. Sanya, China, 2008. 142-147.
[12]	WEI Q . CDRM: a cost-effective dynamic replication management scheme for cloud storage cluster[A]. Proceeding of the IEEE International Confe-rence on Cluster Computing[C]. Heraklion, Greece, 2010. 88-193.
[13]	WANG Z , LI T , XIONG N , et al . A novel dynamic network data rep-lication scheme based on historical access record and ve dele-tion[J]. The Journal of Supercomputing, 2012,62(1):227-250.
[14]	KHAN S U , AHMAD I . A pure Nash equilibrium-based game theoret-ical method for data replication across multiple servers[J]. IEEE Transactions on Knowledge and Data Engineering, 2009,21(4):537-553.
[15]	GHEMAWAT S , GOBIOFF H , LEUNG S T . The google file system[A]. Proceedings of the 19th ACM Symposium on Operating Systems Prin-ciples[C]. Bolton, UK, 2003. 29-43.
[16]	TFS(taobao file system)[EB/OL].
[17]	HDFS(hadoop distributed file system)[EB/OL].
[18]	QU Y , XIONG N . RFH: a resilient, fault-tolerant and high-efficient replication algorithm for distributed cloud storage[A]. Proceedings of the 41st International Conference on Parallel Processing[C]. Pittsburgh, USA, 2012. 520-529.
[19]	ANANTHANARAYANAN G , AGARWAL S , KANDULA S , et al . Scarlett: coping with skewed content popularity in mapreduce clus-ters[A]. Proceedings of the 6th Conference on Computer Systems[C]. Salzburg, Austria, 2011. 287-300.
[20]	陈康, 余宏亮, 张堃．对等网络中基于位置信息和文件流行度的自适应复本管理算法[J]．计算机学报， 2009,32(10):1927-1937. CHEN K , YU H L , ZHANG K . Adaptive replication manageme t algorithm based on location and file popularity for peer-to-peer network[J]. Chinese Journal of Computers, 2009,32(10):1927-1937.
[21]	KAVALANEKAR S , WORTHINGTON B , ZHANG Q , et al . Charac-terization of storage workload traces from production indows serv-ers[A]. Proceedings of IEEE International Symposium on Workload Characterization[C]. Seattle, USA, 2008. 119-128.
[22]	YADWADKAR N J , BHATTACHARYYA C , GOPINATH K , et al . Discovery of application workloads from network file traces[A]. Pro-ceedings of the 8th USENIX Conference on File and Storage Tech-nologies[C]. San Jose, USA, 2010. 183-196.
[23]	OSTROWSKI J R , SARHAN N J . Characterization of social video[A]. Proceedings of IS＆T/SPIE Electronic Imaging[C]. San Jose, USA, 2009. 347-356.
[24]	GAO S , WANG H , WANG J . Research and application of web caching workload characteristics model[A]. Proceedings of the 2nd IEEE In-ternational Conference on Information Management and E gineer-ing[C]. Chengdu, China, 2010. 105-109.
[25]	CHOI J , REAZ A S , MUKHERJEE B . A survey of user behavior in VoD service and bandwidth-saving multicast streaming schemes[J]. IEEE Communications Surveys ＆ Tutorials, 2012,14(1):156-169.
[26]	YU H , ZHENG D , ZHAO B Y , et al . Understanding user behavior in large-scale video-on-demand systems[A]. EuroSys'06[C]. Leuven, Belgium, 2006. 333-344.