网络计算环境分布式COW盘构建方法

doi:1000-436X(2012)07-0036-08

摘要/Abstract

摘要：

针对大规模网络计算环境的分布式计算和数据集中存储特点，结合COW(copy-on-write)磁盘记录改写块的稀疏和突发特性，提出了一种分布式COW网络盘体系结构，将服务器处理所有客户主机相应COW盘的聚合开销分摊到各个客户主机自身，以加速网络计算环境系统构建速度；设计了一种改进的64bit位图压缩算法以有效减少COW盘位图文件大小，节省服务器磁盘空间和降低分布式COW盘网络传输开销；提出一种适合COW盘改写块突发特性的预取算法，以提高分布式COW盘相应Cache的命中率。实验结果表明基于改进的COW位图压缩和预取算法实现的分布式COW盘降低了多客户主机网络计算环境构建延迟。

关键词: 网络计算, 分布式COW盘, 位图压缩, 预取

Abstract:

Aiming to distributed computing and data centralizing storage in large-scale network computing environments and combining sparse and bursting characteristic of COW(copy-on-write) disk storing modified data blocks,the distributed COW network disk architecture was presented.The system building process of network computing environments was speeded up by apportioning the aggregation spending that server manages all related COW disks of client hosts to each client host.A improving 64bit bitmap compress arithmetic was designed to reduce the size of COW disk’s bitmap,so server disk space was saved and the network communication spending of distributed COW disk was less.The modified prefetching arithmetic based on COW bursting blocks was presented to improve the hit rate of Cache relating with distributed COW disk.Experiments on the prototype show that the distributed COW mechanism can effectively reduce system building latency of multi-client hosts network computing environment based on improving COW bitmap compress and prefetching arithmetic.

Key words: network computing, distributed COW disk, bitmap compress, prefetch

谭怀亮,罗政,贺再红,李仁发. 网络计算环境分布式COW盘构建方法[J]. 通信学报, 2012, 33(7): 36-43.

Huai-liang TAN,Zheng LUO,Zai-hong HE,Ren-fa LI. Building method of distributed COW disk on network computing environment[J]. Journal on Communications, 2012, 33(7): 36-43.

图/表 7

图1

表1

图2

图3

图4

图5

表2

参考文献 22

[1]	SAHA D , MUKHERJEE A . Pervasive computing:a paradigm for the 21st century[J]. IEEE Computer, 2003,36(3): 25-31.
[2]	马一力, 傅湘林, 韩晓明 . 存储与计算的分离[J]. 计算机研究与发展, 2005,42(3): 520-530. MA Y L , FU X L , HAN X M . The separation between storage and computation[J]. Journal of Computer Research and Development, 2005,42(3): 520-530.
[3]	HENTWICH R G , KAPPNER T . Network computers-ubiquitous computing or dumb multimedia[A]. The 3rd Int’ l Symp on Autonomous Decent Ralized Systems[C]. Berlin, 1997.
[4]	COMERFORD R . The battle for the desktop[J]. IEEE Spectrum, 1997,34(5): 20-28.
[5]	RICHARDSON T , STAFFORD F Q , WOOD K R . Virtual network computing[J]. IEEE International Computing, 1998,2(1): 33-38.
[6]	INTEL INC . Preboot execution environment (PXE) specification[EB/OL]. , 1998.
[7]	ZHANG Y X . Transparence computing:concept,architecture and implementation[J]. Chinese Journal of Electronics, 2004,32(12A): 169-174.
[8]	YIN Y , LIU Z J , TANG H Y . SonD:a fast service deployment system based on IP SAN[A]. IPDPS[C]. Miami,FL, 2008. 1-10.
[9]	SAPUNTZAKIS C , CHANDRA R , PFAFF B . Optimizing the migration of virtual computers[A]. Proc of the 5th Symp on Operating Systems Design and Implementation[C]. Boston, 2002. 377-390.
[10]	BELLARD F . QEMU,a fast and portable dynamic translator[A]. Proc of the USENIX Annual Technical Conf(USENIX 2005)[C]. Berkeley:USENIX Association, 2005. 41-46.
[11]	陈彬, 肖侬, 蔡志平 . 基于优化COW虚拟块设备的虚拟机按需部署机制[J]. 计算机学报, 2009,32(10): 1915-1926. CHEN B , XIAO N , CAI Z P . On-demand deployment of virtual machines based on optimized COW virtual block device[J]. Chinese Journal of Computers, 2009,32(10): 1915-1926.
[12]	ZACHARY P , RANDAL B . Ext3cow:a time-shifting file system for regulatory compliance[J]. ACM Transactions on Storage, 2005,1(2): 190-212.
[13]	MONTERIRO J , DALLE O . CORRAL:stackable copy-on-write versioning device using Linux device-mapper[A]. USENIX Annual Technical Conference(USENIX'08)[C]. Poster, 2008.
[14]	WU K , OTOO E J , SHOSHANI A . Optimizing bitmap indices with efficient compression[J]. ACM Transactions on Database System, 2006,31(1): 1-38.
[15]	MICHAL S , ROBERT W . RLH:bitmap compression technique based on run-length and Huffman encoding[J]. Information Systems, 2009,34(4,5): 400-414.
[16]	ALESSANDRO C , ROBERTO D P . Concise:compressed’n’ composable integer set[J]. Information Processing Letters, 2010,110(16): 644-650.
[17]	SMITH A J . Cache memories[J]. ACM Computer, 1982,14(3): 473-530.
[18]	JOHN W C F , JANAK H P . Data prefetching in multiprocessor vector cache memories[A]. Proceedings of the 18th Annual International Symposium on Computer Architecture[C]. Toronto, 1991. 54-63.
[19]	谢学军, 叶以正, 邱善勤 . 基于马尔可夫模型的数据值预取方案[J]. 电子学报, 2007,35(2): 307-310. XIE X J , YE Y Z , QIU S Q . Data value prefetching method based on markov model[J]. Chinese Journal of Electronics, 2007,35(2): 307-310.
[20]	谭怀亮, 朱存望, 张镇平 . iSCSI网络计算模式下的可靠多播策略[J]. 计算机研究与发展, 2011,48(9): 216-223. TAN H L , ZHU C W , ZHANG Z P . The reliable multicast method on iSCSI network computer[J]. Journal of Computer Research and Development, 2011,48(9): 216-223.
[21]	谭怀亮, 尹斌 . 支持IP-SAN远程映射与引导的固件协议栈[J]. 通信学报, 2009,30(10): 58-67. TAN H L , YIN B . Firmware protocol stack supporting remote boot and storage volumes mapping in IP-SAN[J]. Journal on Communications, 2009,30(10): 58-67.
[22]	谭怀亮, 王燕, 孙建华 . 分布式系统卷重构过程的改写块预取方法[J]. 湖南大学学报, 2009,36(1): 77-80. TAN H L , WANG Y , SUN J H . Prefetch method for rewrite block in the rebuilding process of distributed system volumes[J]. Journal on Hunan University, 2009,36(1): 77-80.

填充字	类型	突发改写比特起始比特	改写比特比特数	合并的填充字计数器
1	0或1	6bit	6bit	50bit

类型	COW盘位图文件大小/Mbit	百兆比特每秒网络传输延迟/ms	吉比特每秒网络传输延迟/ms
未压缩	20	260	35
压缩	2.1	28	11