分布式存储系统中的数据高效缓存方法

doi:10.11959/j.issn.2096-0271.2021018

大数据 ›› 2021, Vol. 7 ›› Issue (2): 147-157.doi: 10.11959/j.issn.2096-0271.2021018

• 专题：高性能计算虚拟数据空间 • 上一篇下一篇

分布式存储系统中的数据高效缓存方法

杨青霖, 吴桂勇, 张广艳

清华大学计算机科学与技术系，北京 100084

出版日期:2021-03-15 发布日期:2021-03-01
作者简介:杨青霖（1995- ），男，清华大学计算机科学与技术系博士生，主要研究方向为分布式系统、纠删码存储。
吴桂勇（1992- ），男，清华大学计算机科学与技术系硕士生，主要研究方向为分布式系统、存储系统。
张广艳（1976- ），男，博士，清华大学计算机系长聘副教授、博士生导师，主要从事大数据存储与分析的理论和方法研究，包括大数据计算、存储系统与分布式处理等方面。研究得到了国家杰出青年科学基金项目、国家重点研发计划项目、国家973项目和国家863项目等的支持。近年来提出了大规模存储系统构建及访问的方法与关键技术，有效提高了存储系统的性能、扩展性和可用性。发表学术论文40余篇，其中在FAST、USENIX ATC、ACM TOS、IEEE TC、IEEE TPDS等计算机系统领域高水平国际会议和期刊发表论文20余篇。近五年以第一发明人获得美国发明专利授权1项、中国发明专利授权7项。
基金资助:
国家重点研发计划资助项目(2018YFB0203902);国家自然科学基金资助项目(62025203)

An approach to buffering data efficiently in distributed storage systems

Qinglin YANG, Guiyong WU, Guangyan ZHANG

Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China

Online:2021-03-15 Published:2021-03-01
Supported by:
The National Key Research and Development Program of China(2018YFB0203902);The National Natural Science Foundation of China(62025203)

摘要/Abstract

摘要：

针对典型分布式存储系统存在的写放大、I/O路径过长、响应时延较高等问题，提出了一种基于SSD的分布式存储系统中数据高效缓存方法，采用读写旁路和懒惰缓存的缓存管理策略，以及兼顾最近访问时间和历史访问频率的缓存替换策略，并根据前台工作负载的变化情况，自适应地调整主动回刷脏数据的速率，显著提升了存储系统的读写性能。

关键词: 分布式存储, 分布式数据缓存, 读写旁路, 懒惰缓存, 替换策略

Abstract:

To address the problems of write amplification, long I/O path, and high access latency in distributed storage systems, an efficient SSD-based caching approach for distributed storage systems was proposed.This approach adopts read/write bypassing and lazy caching methods to manage the cache system, considers last access time and historical access frequency when performing cache replacement, and adjusts the flushing speed according to the foreground workload.It improves significantly the reading and writing performance of storage systems.

Key words: distributed storage, distributed data caching, read/write bypassing, lazy caching, replacement strategy

中图分类号:

TP333

杨青霖, 吴桂勇, 张广艳. 分布式存储系统中的数据高效缓存方法[J]. 大数据, 2021, 7(2): 147-157.

Qinglin YANG, Guiyong WU, Guangyan ZHANG. An approach to buffering data efficiently in distributed storage systems[J]. Big Data Research, 2021, 7(2): 147-157.

图/表 8

图1

图2

图3

图4

图5

图6

图7

表1

播放真实trace负载下的I/O性能"

trace名称	测量指标	Cache Tiering	HOSD	加速比
prxy_1	读IOPS	2 248	2 542	1.13
	写IOPS	1 185	1 341	1.13
wdev_0	读IOPS	343	373	1.09
	写IOPS	1 326	1 469	1.11
prxy_0	写IOPS	1 568	1 668	1.06
webmail	读IOPS	525	570	1.09
	写IOPS	1 143	1 159	1.01
hm_0	读IOPS	425	438	1.03
	写IOPS	1 079	1 122	1.04
几何平均值				$1 . 08$

表1

参考文献 20

[1]	陈游旻, 李飞, 舒继武 . 大数据环境下的存储系统构建：挑战、方法和趋势[J]. 大数据, 2019,5(4): 27-40.
	CHEN Y M , LI F , SHU J W . Building storage systems in big data era:challenges,methods and trends[J]. Big Data Research, 2019,5(4): 27-40.
[2]	LEVENTHAL A . Flash storage memory[J]. Communications of the ACM, 2008,51(7): 47-51.
[3]	PETERS M . NetApps solid state hierarchy[Z]. ESG White Paper, 2009.
[4]	KGIL T , ROBERTS D , MUDGE T . Improving NAND flash based disk caches[C]// The 35th Annual International Symposium on Computer Architecture. Piscataway:IEEE Press, 2008: 327-338.
[5]	MESNIER M P , AKERS J B . Differentiated storage services[C]// The 23rd ACM Symposium on Operating Systems Principles. New York:ACM Press, 2011: 57-70.
[6]	PRITCHETT T , THOTTETHODI M . SieveStore:a highly-selective,ensemblelevel disk cache for cost-performance[J]. ACM SIGARCH Computer Architecture News, 2010,38(3): 163-174.
[7]	OH Y , CHOI J , LEE D ,et al. Caching less for better performance:balancing cache size and update cost of flash memory cache in hybrid storage systems[C]// The 10th USENIX Conference on File and Storage Technologies. New York:ACM Press, 2012,25.
[8]	CANIM M , MIHAILA G A , BHATTACHARJEE B ,et al. SSD bufferpool extensions for database systems[J]. Proceedings of the VLDB Endowment, 2010,3(1-2): 1435-1446.
[9]	DO J , ZHANG D H , PATEL J M ,et al. Turbocharging DBMS buffer pool using SSDs[C]// The 2011 International Conference on Management of Data. New York:ACM Press, 2011: 1113-1124.
[10]	GUERRA J , PUCHA H , GLIDER J ,et al. Cost effective storage using extent based dynamic tiering[C]// The 9th USENIX Conference on File and Storage Technologies. Berkeley:USENIX Association, 2011: 20.
[11]	CHEN F , KOUFATY D A , ZHANG X D . Hystor:making the best use of solid state drives in high performance storage systems[C]// The International Conference on Supercomputing. New York:ACM Press, 2011: 22-32.
[12]	APPUSWAMY R , VAN MOOLENBROEK D C , TANENBAUM A S . Integrating flash-based SSDs into the storage stack[C]// 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies. Piscataway:IEEE Press, 2012: 1-12.
[13]	WANG A I A , REIHER P L , POPEK G J ,et al. Con-quest:better performance through a disk/persistent-RAM hybrid file system[C]// The 2002 USENIX Annual Technical Conference. New York:ACM Press, 2002.
[14]	DEBNATH B , SENGUPTA S , LI J . ChunkStash:speeding up inline storage deduplication using flash memory[C]// The 2010 USENIX Conference on USENIX Annual Technical Conference. Berkeley:USENIX Association, 2010:16.
[15]	YANG Q , REN J . I-CASH:intelligently coupled array of SSD and HDD[C]// 2011 IEEE 17th International Symposium on High Performance Computer Architecture. Piscataway:IEEE Press, 2011: 278-289.
[16]	WANG S C , LU Z Y , CAO Q ,et al. BCW:buffer-controlled writes to HDDs for SSD-HDD hybrid storage server[C]// The 2020 USENIX Conference on File and Storage Technologies. Berkeley:USENIX Association, 2020: 253-266.
[17]	LIU Z X , BAI Z H , LIU Z M ,et al. DistCache:provable load balancing for large-scale storage systems with distributed caching[C]// The 17th USENIX Conference on File and Storage Technologies. Berkeley:USENIX Association, 2019: 143-157.
[18]	ZHANG Y , HUANG P , ZHOU K ,et al. OSCA:an online-model based cache allocation scheme in cloud block storage systems[C]// The 2020 USENIX Conference on USENIX Annual Technical Conference. Berkeley:USENIX Association, 2020: 785-798.
[19]	BERGER D S , BERG B , ZHU T ,et al. RobinHood:tail latency aware caching dynamic reallocation from cache-rich to cache-poor[C]// The 13th USENIX Symposium on Operating Systems Design and Implementation. Berkeley:USENIX Association, 2018: 195-212.
[20]	LUO T Q , AGGARWAL V , PELEATO B . Coded caching with distributed storage[J]. IEEE Transactions on Information Theory， 2016： 99.

分布式存储系统中的数据高效缓存方法

An approach to buffering data efficiently in distributed storage systems

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 20

相关文章 2

Metrics

推荐阅读 0

[1]	邓镇龙, 陈志广. 面向非易失内存的MPI-IO接口优化[J]. 大数据, 2021, 7(2): 172-181.
[2]	秦广军, 肖利民, 张广艳, 牛北方, 陈志广. 面向国家高性能计算环境的虚拟数据空间系统[J]. 大数据, 2021, 7(2): 101-122.