面向非易失内存的MPI-IO接口优化

doi:10.11959/j.issn.2096-0271.2021020

Abstract

Abstract:

In an HPC system where multiple computation nodes of an MPI application simultaneously access files in underlying storage systems, the I/O overhead is affected by the access mode and the properties of external storage devices.Based on the patterns of MPI applications to access files, an optimization for MPI-IO interface for persistent memories was introduced on high-bandwidth, low-latency, byte-addressable, data-persistent memories.By constructing distributed data cache, maintaining persistent metadata and leveraging optimizations on data movements among processes, applications were enabled to efficiently manage and utilize persistent memories with data consistency guaranteed, resulting in tens of times improvement on read/write bandwidth.Further optimizations on parallelism were set for future work.

Key words: non-volatile memory, MPI-IO, distributed data cache

CLC Number:

TP311

Zhenlong DENG, Zhiguang CHEN. An optimization of MPI-IO interface for non-volatile memory[J]. Big Data Research, 2021, 7(2): 172-181.

Figures/Tables 5

References 18

[1]	THAKUR R , GROPP W , LUSK E ,et al. On implementing MPI-IO portably and with high performance[C]// The 6th Workshop on I/O in Parallel and Distributed Systems. New York:ACM Press, 1999: 23-32.
[2]	THAKUR R , GROPP W , LUSK E ,et al. Data sieving and collective I/O in ROMIO[C]// The 7th Symposium on the Frontiers of Massively Parallel Computation. Piscataway:IEEE Press, 1999: 182-189.
[3]	SANKARAN S , SQUYRES J M , BARRETT B ,et al. The LAM/MPI checkpoint/restart framework:system-initiated checkpointing[J]. The International Journal of High Performance Computing Applications, 2005,19(4): 479-493.
[4]	IZRAELEVITZ J , YANG J , ZHANG L ,et al. Basic performance measurements of the Intel Optane DC persistent memory module[J]. arXiv preprint,2019,arXiv:1903.05714.
[5]	陈游旻, 李飞, 舒继武 . 大数据环境下的存储系统构建:挑战、方法和趋势[J]. 大数据, 2019,5(4): 27-40.
	CHEN Y M , LI F , SHU J W . Building storage systems in big data era:challenges,methods and trends[J]. Big Data Research, 2019,5(4): 27-40.
[6]	VOLOS H , TACK A J , SWIFT M M ,et al. Mnemosyne:lightweight persistent memory[J]. ACM SIGARCH Computer Architecture News, 2011,39(1): 91-104.
[7]	COBURN J , CAULFIELD A M , AKEL A ,et al. NV-Heaps:making persistent objects fast and safe with next-generation,non-volatile memories[J]. ACM SIGARCH Computer Architecture News, 2011,39(1): 105-118.
[8]	WU X J , QIU S , REDDY A L N ,et al. SCMFS:a file system for storage class memory and its extensions[J]. ACM Transactions on Storage, 2013,9(3).
[9]	CONDIT J , NIGHTINGALE E B , FROST C ,et al. Better I/O through byteaddressable,persistent memory[C]// The ACM SIGOPS 22nd Symposium on Operating Systems Principles. New York:ACM Press, 2009: 133-146.
[10]	CHEN Y M , SHU J W , OU J X ,et al. HiNFS:a persistent memory file system with both buffering and direct-access[J]. ACM Transactions on Storage, 2018,14(1): 1-30.
[11]	XU J , SWANSON S . NOVA:a logstructured file system for hybrid volatile/non-volatile main memories[C]// The 14th USENIX Conference on File and Storage Technologies.[S.l.:s.n.], 2016: 323-338.
[12]	XU J , ZHANG L , MEMARIPOUR A ,et al. NOVA-Fortis:a fault-tolerant non-volatile main memory file system[C]// The 26th Symposium on Operating Systems Principles. New York:ACM Press, 2017: 478-496.
[13]	ZUO P F , HUA Y . A write-friendly and cache-optimized hashing scheme for non-volatile memory systems[J]. IEEE Transactions on Parallel and Distributed Systems, 2018,29(5): 985-998.
[14]	ZUO P F , HUA Y , WU J . Level Hashing:a high-performance and flexible-resizing persistent hashing index structure[J]. ACM Transactions on Storage, 2019,15(2).
[15]	LU Y Y , SHU J W , CHEN Y M ,et al. Octopus:an RDMA-enabled distributed persistent memory file system[C]// The 2017 USENIX Annual Technical Conference. New York:ACM Press, 2017: 773-785.
[16]	YANG J , IZRAELEVITZ J , SWANSON S . Orion:a distributed file system for nonvolatile main memories and RDMAcapable networks[C]// The 17th USENIX Conference on File and Storage Technologies. New York:ACM Press, 2019: 221-234.
[17]	RAHMAN M W , ISLAM N S , LU X ,et al. NVMD:non-volatile memory assisted design for accelerating MapReduce and DAG execution frameworks on HPC systems[C]// 2017 IEEE International Conference on Big Data. Piscataway:IEEE Press, 2017: 369-374.
[18]	吴昊, 陈康, 武永卫 ,等. 基于RDMA和NVM的大数据系统一致性协议研究[J]. 大数据, 2019,5(4): 89-99.
	WU H , CHEN K , WU Y W ,et al. Research on the consensus of big data systems based on RDMA and NVM[J]. Big Data Research, 2019,5(4): 89-99.

Metrics

Recommended 0

No Suggested Reading articles found!

An optimization of MPI-IO interface for non-volatile memory

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 18

Related Articles 4

Metrics

Recommended 0

[1]	Yongfeng WANG, Zhiguang CHEN. A survey of persistent index data structures on non-volatile memory [J]. Big Data Research, 2021, 7(6): 78-88.
[2]	Youmin CHEN, Fei LI, Jiwu SHU. Building storage systems in big data era:challenges,methods and trends [J]. Big Data Research, 2019, 5(4): 27-40.
[3]	Xiaoyuan WANG, Xiaofei LIAO, Haikun LIU, Hai JIN. Big data oriented hybrid memory systems [J]. Big Data Research, 2018, 4(4): 15-34.
[4]	Xin LI, Xuan CHEN, Zhiqiu HUANG. Analysis on hybrid memory architecture for big data application [J]. Big Data Research, 2018, 4(3): 61-80.