广域虚拟数据空间中边缘缓存系统的研究与实现

doi:10.11959/j.issn.2096-0271.2021049

Abstract

Abstract:

Aiming at the problem of a large amount of network bandwidth wasted by data redundancy transmission when the edge client accesses and shares remote data in the GVDS, the caching technology in the wide-area virtual data space system was studied and an edge caching mechanism to optimize the data access path was proposed.The data is cached at a file granularity close to the edge client, thereby improving the performance of upper-level applications to access and share data.The experimental results show that the edge cache system proposed can improve the performance of wide-area data sharing as a supplement to the GVDS.

Key words: wide area network, edge cache, global virtual data space, shared data access performance

CLC Number:

TP319

Jiantong HUO, Limin XIAO, Zhisheng HUO, Yaowen XU. Research and implementation of edge cache system in global virtual data space across WAN[J]. Big Data Research, 2021, 7(5): 57-81.

Figures/Tables 27

References 16

[1]	肖利民, 宋尧, 秦广军 ,等. GVDS:面向广域高性能计算环境的虚拟数据空间[J]. 大数据, 2021,7(2): 123-146.
	XIAO L M , SONG Y , QIN G J ,et al. GVDS:a global virtual data space for wide-area high-performance computing environments[J]. Big Data Research, 2021,7(2): 123-146.
[2]	ASHTON K . That “Internet of things”thing[J]. RFID Journal, 2009,22(7): 97-114.
[3]	施巍松, 张星洲, 王一帆 ,等. 边缘计算:现状与展望[J]. 计算机研究与发展, 2019,56(1): 69-89.
	SHI W S , ZHANG X Z , WANG Y F ,et al. Edge computing:state-of-the-art and future directions[J]. Journal of Computer Research and Development, 2019,56(1): 69-89.
[4]	刘立明, 王彬 . 气象网格环境下大数据的端到端传输机制研究[J]. 计算技术与自动化, 2014,33(1): 122-126.
	LIU L M , WANG B . Research of an endto-end transfer mechanism for big data in CMAGrid environment[J]. Computing Technology and Automation, 2014,33(1): 122-126.
[5]	王彬, 宗翔, 田浩 . 国家气象计算网格的设计与建立[J]. 应用气象学报, 2010,21(5): 632-640.
	WANG B , ZONG X , TIAN H . Design and establishment of a nationwide meteorological computational grid[J]. Journal of Applied Meteorological Science, 2010,21(5): 632-640.
[6]	European open science cloud[J]. Nature Genetics, 2016,48(8): 821.
[7]	DILLEY J , MAGGS B , PARIKH J ,et al. Globally distributed content delivery[J]. IEEE Internet Computing, 2002,6(5): 50-58.
[8]	RAMASWAMY L , LIU L , IYENGAR A . Scalable delivery of dynamic content using a cooperative edge cache grid[J]. IEEE Transactions on Knowledge and Data Engineering, 2007,19(5): 614-630.
[9]	SATYANARAYANAN M , BAHL P , CACERES R ,et al. The case for VMbased cloudlets in mobile computing[J]. IEEE Pervasive Computing, 2009,8(4): 14-23.
[10]	WILKINSON S STORJ a peer-to-peer cloud storage network[Z]. 2014.
[11]	CHEN B Q , YANG C Y , WANG G . Highthroughput opportunistic cooperative device-to-device communications with caching[J]. IEEE Transactions on Vehicular Technology, 2017,66(8): 7527-7539.
[12]	TAN H S , JIANG S H C , HAN Z H ,et al. Camul:online caching on multiple caches with relaying and bypassing[C]// Proceedings of IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2019: 244-252.
[13]	CHERKASOVA L . Improving WWW proxies performance with greedy-dualsize-frequency caching policy[Z]. 1998.
[14]	KARGER D , LEHMAN E , LEIGHTON T ,et al. Consistent hashing and random trees:distributed caching protocols for relieving hot spots on the World Wide Web[C]// Proceedings of the 29th Annual ACM Symposium on Theory of Computing. New York:ACM Press, 1997: 654-663.
[15]	DAS A , GUPTA I , MOTIVALA A . SWIM:scalable weakly-consistent infectionstyle process group membership protocol[C]// Proceedings of International Conference on Dependable Systems and Networks. Piscataway:IEEE Press, 2002: 303-312.
[16]	BIRMAN K . The promise,and limitations,of gossip protocols[J]. ACM SIGOPS Operating Systems Review, 2007,41(5): 8-13.

Metrics

Recommended 0

No Suggested Reading articles found!

类型	共享锁（S）	共享排他锁（SX）	排他锁（X）
共享锁（S）	兼容	兼容	冲突
共享排他锁（SX）	兼容	冲突	冲突
排他锁（X）	冲突	冲突	冲突

接口	接口规范	返回值	接口说明
缓存文件添加接口	POST /cache Json Value/r/n	Value 200 OK；500 Internal Server Error	管理接口，表示向边缘缓存添加指定全局路径对应的文件
缓存文件查询接口	POST /cache Json Value/r/n	Value 200 OK；500 Internal Server Error	管理接口，表示向边缘缓存查询指定全局路径对应的文件
缓存文件更新接口	POST /cache Json Value/r/n	Value 200 OK；500 Internal Server Error	管理接口，表示同步边缘缓存中指定全局路径对应的文件
缓存文件删除接口	POST /cache Json Value/r/n	200 OK；500 Internal Server Error	管理接口，表示删除边缘缓存中指定全局路径对应的文件
边缘缓存集群状态查询接口	GET /cluster	Value 200 OK；500 Internal Server Error	管理接口，表示获取边缘缓存服务节点集群状态信息
边缘缓存容量状态查询接口	GET /storage	Value 200 OK；500 Internal Server Error	查询边缘缓存、存储容量状态

接口	接口规范	参数说明
缓存元数据查询接口	CacheMetaEdgeCacheSearch(	global_path：文件全局路径；
	string global_path,	ioproxy_info：表示负责该文件的I/O代理信息；
	string ioproxy_info,	relative_path：I/O代理相对路径；
	string relative_path,	token：客户凭证；
	string token	return：缓存元数据信息；失败
	)
缓存文件写入接口	Int64 EdgeCacheWrite(	global_path：文件全局路径；
	string global_path,	ioproxy_info：表示负责该文件的I/O代理信息；
	string ioproxy_info,	buf：文件数据存储缓冲；
	string relative_path,	offset：文件写入起始位置；
	byte[]buf,	length：文件写入数据长度；
	int64 offset,	return：写入的数据量值（写入成功），-1（写入失败）
	int64 length
	)
缓存文件读取接口	Int64 EdgeCacheRead(	global_path：文件全局路径；
	string global_path,	ioproxy_info：表示负责该文件的I/O代理信息；
	string ioproxy_info	relative_path：I/O代理相对路径；
	string relative_path,	buf：文件数据存储缓冲；
	byte[]buf,	offset：文件读取起始位置；
	int64 offset,	length：文件读取数据长度；
	int64 length	return：读取的数据量值（读取成功），-1（读取失败）
	)
缓存文件关闭接口	Int64 EdgeCacheClose(	global_path：文件全局路径；
	string global_path,	ioproxy_info：表示负责该文件的I/O代理信息；
	string ioproxy_info,	relative_path：I/O代理相对路径；
	string relative_path	return：0（关闭成功），-1（关闭失败）
	)

网络环境	上行带宽	下行带宽	时延	抖动	丢包率
回环网络	7.8 GB/s	7.9 MB/s	0.04 ms	0.01 ms	0
局域网	117.8 MB/s	117.8 MB/s	0.12 ms	0.02 ms	0
广域网	4.6 MB/s	7.3 MB/s	21.22 ms	2.29 ms	1%

Research and implementation of edge cache system in global virtual data space across WAN

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 27

References 16

Related Articles 2

Metrics

Recommended 0

[1]	Chenhao ZHANG, Limin XIAO, Guangjun QIN, Yao SONG, Shixuan JIANG, Jiye WANG. A wide-area collaborative scheduling system oriented to big data processing applications [J]. Big Data Research, 2021, 7(5): 82-97.
[2]	Limin XIAO, Yao SONG, Guangjun QIN, Hanjie ZHOU, Chaobo WANG, Bing WEI, Wei WEI, Zhisheng HUO. GVDS: a global virtual data space for wide-area high-performance computing environments [J]. Big Data Research, 2021, 7(2): 123-146.