一种软硬件结合的大数据访存踪迹收集分析工具集

doi:10.11959/j.issn.2096-0271.2019031

大数据 ›› 2019, Vol. 5 ›› Issue (4): 41-49.doi: 10.11959/j.issn.2096-0271.2019031

• 专题：大数据的系统结构 • 上一篇下一篇

一种软硬件结合的大数据访存踪迹收集分析工具集

李作骏^1,²,潘海洋^1,²,陈明宇^1,²,包云岗^1,²

¹ 中国科学院大学，北京 100049
² 中国科学院计算技术研究所先进计算机系统研究中心，北京 100190

出版日期:2019-07-15 发布日期:2019-08-09
作者简介:李作骏（1990- ），男，中国科学院大学博士生，主要研究方向为计算机系统结构。|潘海洋（1990- ），男，中国科学院大学博士生，主要研究方向为计算机系统结构。|陈明宇（1972- ），男，博士，中国科学院计算技术研究所研究员，主要研究方向为高性能计算机体系结构、操作系统及并行算法优化。|包云岗（1980- ），男，博士，中国科学院计算技术研究所研究员，计算机体系结构国家重点实验室教授，先进计算机系统研究中心主任，中国科学院大学岗位教授。担任中国计算机学会理事、普及工作委员会主任，中国科学院青年创新促进会理事。主要研究方向为计算机系统结构，主持研制多款达到国际先进水平的系统，在国际会议期刊发表了40余篇论文，相关技术在华为技术有限公司、阿里巴巴集团、英特尔公司等国内外企业应用，多次受邀担任ASPLOS、ISCA、MICRO、SC等国际会议的程序委员会委员。
基金资助:
“十三五”国家重点研发计划基金资助项目(2017YFB1001602)

A hybrid memory trace collection and analysis toolkit for big data applications

Zuojun LI^1,²,Haiyang PAN^1,²,Mingyu CHEN^1,²,Yungang BAO^1,²

¹ University of Chinese Academy of Sciences,Beijing 100049,China
² Center for Advanced Computer Systems,Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China

Online:2019-07-15 Published:2019-08-09
Supported by:
The National Key Research and Development Program During the Thirteenth Five-Year Plan Period(2017YFB1001602)

摘要/Abstract

摘要：

以Spark为代表的内存计算框架的兴起、新型非易失性内存研究的逐步深入以及数据安全形势的日益严峻，使得现有的访存行为分析工具无法满足对大数据应用访存行为进行分析的需求。提出了一种软硬件结合的大数据访存踪迹收集分析工具集，在由硬件收集的基本访存踪迹的基础上，结合软件信息同步及离线标注的方式，可以高速、准确、无失真地获取具备丰富语义信息的访存行为信息，且为大数据访存的实时安全监控提供了一种实现方式。最后，通过实验对一组真实的大数据应用进行了访存踪迹采集和分析。

关键词: 访存踪迹, 访存行为, 大数据

Abstract:

The rise of in-memory computing framework represented by Spark,the gradual deepening of new non-volatile memory research and the increasingly severe data security situation made the existing memory behavior analysis tools unable to meet the demand for big data applications.A software-hardware hybrid memory trace collection and analysis toolkit for big data applications was proposed.Based on the basic memory trace collected by hardware,the memory behavior information with rich semantic information can be obtained quickly,accurately and undistorted by combining software information synchronization and offline annotation.It also provides an implementation method for real-time security monitoring of large data access.Finally,a group of real big data applications were analyzed by this toolkit.

Key words: memory trace, memory behavior, big data

中图分类号:

TP391

李作骏,潘海洋,陈明宇,包云岗. 一种软硬件结合的大数据访存踪迹收集分析工具集[J]. 大数据, 2019, 5(4): 41-49.

Zuojun LI,Haiyang PAN,Mingyu CHEN,Yungang BAO. A hybrid memory trace collection and analysis toolkit for big data applications[J]. Big Data Research, 2019, 5(4): 41-49.

图/表 6

表1

图1

表2

表3

表4

图2

参考文献 33

[1]	ZAHARIA M , CHOWDHURY M , FRANKLIN M J ,et al. Spark:cluster computing with working sets[C]// The 2nd USENIX Conference on Hot Topics in Cloud Computing,June 22-25,2010,Boston,USA. Berkeley:USENIX Association, 2010:10.
[2]	BEZ R , PIROVANO A . Non-volatile memory technologies:emerging concepts and new materials[J]. Materials Science in Semiconductor Processing, 2004,7(4-6): 349-355.
[3]	ROSENFELD P , COOPERBALIS E , JACOB B . DRAMSim2:a cycle accurate memory system simulator[J]. IEEE Computer Architecture Letters, 2011,10(1): 16-19.
[4]	MAGNUSSON P , CHRISTENSSON M , ESKILSON J ,et al. Simics:a full system simulation platform[J]. Computer, 2002,35(2): 50-58.
[5]	BINKERT N , BECKMANN B , BLACK G ,et al. The Gem5 simulator[J]. ACM Sigarch Computer Architecture News, 2011,39(2): 1-7.
[6]	JALEEL A , COHN R , LUK C K ,et al. CMP$im:a pin-based on-the-fly multicore cache simulator[C]// The 4th Annual Workshop on Modeling,Benchmarking and Simulation,June 22,2008,Beijing,China.[S.l.:s.n. ], 2008: 28-36.
[7]	PATEL A , AFRAM F , CHEN S F ,et al. MARSSx86:a full system simulator for multicore x86 CPUs[C]// The 48th ACM/EDAC/IEEE Design Automation Conference,June 5-9,2011,New York,USA. Piscataway:IEEE Press, 2011: 1050-1055.
[8]	LUK C K , COHN R , MUTH R ,et al. Pin:building customized program analysis tools with dynamic instrumentation[C]// The 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation,June 12-15,2005,Chicago,USA. New York:ACM Press, 2005: 190-200.
[9]	NETHERCOTE N , SEWARD J . Valgrind:a framework for heavyweight dynamic binary instrumentation[C]// The 28th ACM SIGPLAN Conference on Programming Language Design and Implementation,June 10-13,2007,San Diego,USA. New York:ACM Press, 2007: 89-100.
[10]	SRIVASTAVA A , EUSTACE A . ATOM:a system for building customized program analysis tools[C]// The ACM SIGPLAN 1994 Conference on Programming Language Design and Implementation,June 20-24,1994,Orlando,USA. New York:ACM Press, 1994: 196-205.
[11]	LAURENZANO M , TIKIR M , CARRINGTON L ,et al. PEBIL:efficient static binary instrumentation for Linux[C]// IEEE International Symposium on Performance Analysis of Systems ＆Software,March 28-30,2010,White Plains,USA. Piscataway:IEEE Press, 2010: 175-183.
[12]	REINDERS J . VTune performance analyzer essentials:measurement and tuning techniques for software developers[M]. Santa Clara: Intel PressPress, 2004: 1-500.
[13]	GREGG B , MAURO J . DTrace:dynamictracing in Oracle Solaris,Mac OS X,and free BSD[M]. Upper Saddle River: Prentice Hall ProfessionalPress, 2011: 1-1152.
[14]	TAN Z X , WATERMAN A , AVIZIENIS R ,et al. RAMP gold:an FPGAbased architecture simulator for multiprocessors[C]// The 47th Design Automation Conference,June 13-18,2010,Anaheim,USA. New York:ACM Press, 2010: 463-468.
[15]	ROTHMAN J , CHEN C . BEE technology overview[C]// International Conference on Embedded Computer Systems,July 16-19,2012,Samos,Greece. Piscataway:IEEE Press, 2012:277.
[16]	NANDA A , MAK K K , SUGAVANAM K ,et al. MemorIES:a programmable,real-time hardware emulation tool for multiprocessor server design[J]. ACM SIGPLAN Notices, 2000,35(11): 37-48.
[17]	CHALAINANONT N , NURVITADHI E , MORRISON R ,et al. Real-time L3 cache simulations using the programmable hardware-assisted cache emulator (PHA$E)[C]// IEEE International Conference on Communications,October 27,2003,Austin,USA. Piscataway:IEEE Press, 2003: 86-95.
[18]	HONG J , NURVITADHI E , LU S L . Design,implementation,and verification of active cache emulator (ACE)[C]// The 2006 ACM/SIGDA 14th International Symposium on Field Programmable Gate Arrays,February 22-24,2006,Monterey,USA. New York:ACM Press, 2006: 63-72.
[19]	YOON H M,PARK G H , LEE K W , et a l . Reconfigurable address collector and flying cache simulator[C]// The HighPerformance Computing on the Information Superhighway,April 28-May 2,1997,Seoul,Korea. Piscataway:IEEE Press, 1997:552.
[20]	阮元, 陈明宇, 包云岗 ,等. 基于硬件的内存trace工具——MTT的设计与实现[J]. 电子学报, 2008,36(8): 1519-1525.
	RUAN Y , CHEN M Y , BAO Y G ,et al. The design and implementation of MIT a hardware-based memory trace tool[J]. Acta Electronica Sinica, 2008,36(8): 1519-1525.
[21]	朱晏 . 内存行为、功耗监控平台的硬件设计[D]. 北京:中国科学院大学, 2011.
	ZHU Y . Hardware design of memory behavior and power monitoring platform[D]. Beijing:University of Chinese Academy of Sciences, 2011.
[22]	李作骏 . 访存踪迹收集工具的逻辑设计与实现[D]. 北京:中国科学院大学, 2016.
	LI Z J . Logical design and implementation of memory trace collection tool[D]. Beijing:University of Chinese Academy of Sciences, 2016.
[23]	李作骏, 陈明宇 . 一种支持DDR4的软硬件结合的访存踪迹收集分析工具集[C]// 全国高性能计算学术年会,2018年10月18-20日,青岛,中国. 北京 :中国计算机学会, 2018: 224-231.
	LI Z J , CHEN M Y . Hybrid memory trace collection and analysis toolkit for DDR4[C]// National Annual Conference on High Performance Computing,October 18-20,2018,Qingdao,China. Beijing:China Computer Federation, 2018: 224-231.
[24]	TANG D , BAO Y G , HU W W ,et al. DMA cache:using on-chip storage to architecturally separate I/O data from CPU data for improving I/O p erformance[C]// The 16th International Symposium on High-Performance Computer Architecture,January 9-14,2010,Bangalore,India. Piscataway:IEEE Press, 2010.
[25]	CHEN L C , CUI Z H , BAO Y G ,et al. A lightweight hybrid hardware/software approach for object-relative memory profiling[C]// The 2012 IE EE International Symposium on Performance Analysis of Systems ＆ Software,April 1-3,2012,New Brunswick,USA. Piscataway:IEEE Press, 2012: 46-57.
[26]	ZHANG J T , LIU Y H , LI H F ,et al. PTAT:an efficient and precise tool for tracing and profiling detailed TLB misses[C]// ACM Transactions on Embedded Computing Systems (TECS),April 24-25,2017,Santa Rosa,USA. Piscataway:IEEE Press, 2017.
[27]	HUANG Y B , CUI Z H , CHEN L C ,et al. HaLock:hardware-assisted lock contention detection in multithreaded applications[C]// The 21st Int ernational Conference on Parallel Architectures and Compilation Techniques,September 19-23,2012,Minneapolis,USA. New York:ACM Press, 2012.
[28]	BAO Y G , CHEN M Y , RUAN Y ,et al. HMTT:a platform independent fullsystem memory trace monitoring system[C]// The 2008 ACM Internationa l Conference on Measurement and Modeling of Computer Systems,June 2-6,2008,Annapolis,USA. New York:ACM Press, 2008: 229-240.
[29]	CUI Z H , ZHU Y , BAO Y G ,et al. A fine-grained component-level power measurement method[C]// 2011 International Green Computing Conferen ce and Workshops,July 25-28,2011,Orlando,USA. Piscataway:IEEE Press, 2011.
[30]	LU T Y , LIU Y H , PAN H Y ,et al. TDV cache:organizing off-chip DRAM cache of NVMM from a fusion perspective[C]// The 35th IEEE International Conference on Computer Design (ICCD),November 5-8,2017,Boston,USA. Piscataway:IEEE Press, 2017.
[31]	HENNING J . SPEC CPU2006 benchmark descriptions[J]. ACM SIGARCH Computer Architecture News, 2006,34(4): 1-17.
[32]	BIENIA C , KUMAR S , SINGH J ,et al. The PARSEC benchmark suite:characterization and architectural implications[C]// The 17th International Conference on Parallel Architectures and Compilation Techniques,October 25-29,2008,Toronto,Canada. New York:ACM Press, 2008.
[33]	WANG L , ZHAN J F , LUO C J ,et al. BigDataBench:a big data benchmark suite from Internet services[C]// IEEE20th International Symposium on High Performance Computer Architecture,February 15-19,2014,Orlando,USA. Piscataway:IEEE Press, 2014.

方法	完整	快速	无失真	丰富语义信息	低成本
软件模拟	√-	×	√	√	√
代码插入	√-	×	×	√-	√
硬件计数器	×	√	√	×	√
硬件仿真	√	√	×	√	×
硬件侦听	√	√	√	×	√-
HMTT	√	√	√	√	√

版本	内存速率	传输协议	传输带宽	其他特性
HMTT 1	DDR-200	千兆以太网	125 MB/s	旁路监听、行为标记
HMTT 2	DDR2-667	千兆以太网	375 MB/s	功耗监控、踪迹压缩、页表信息
HMTT 3	DDR3-800	PCIe Gen1	1 GB/s	内存集成、多板同步、函数、对象及I/O信息
HMTT 4	DDR4-1600	PCIe Gen2	2 GB/s	稳定性、可移植性提升

实验平台	组件	参数
发送机	中央处理器	两个Intel E5-2620V2,，6核心12线程，主频2.1 GHz，三级缓存15 MB
	内存	8条32 GB内存，共256 GB，DDR3-800， 8路交错
	操作系统	CentOS 7
接收机	磁盘阵列	4个1 TB固态硬盘组成4 TB大小的磁盘阵列，最高写入带宽为2 000 MB/s

工作集	使用内存大小/GB	访存踪迹大小/GB	描述
Quicksort	247.9	195	排序
SAP HANA	111.9	362	内存计算数据库
PageRank	253.4	553	搜索引擎
TeraSort	253.2	235	排序
Bayes	206.8	96	分类
Kmeans	95.2	425	聚类

一种软硬件结合的大数据访存踪迹收集分析工具集

A hybrid memory trace collection and analysis toolkit for big data applications

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 33

相关文章 15

Metrics

推荐阅读 0

[1]	钱海红, 王茂异, 熊贇. 高等教育数字化转型的现状与发展研究[J]. 大数据, 2023, 9(3): 56-70.
[2]	梅宏, 杜小勇, 金海, 程学旗, 柴云鹏, 石宣化, 靳小龙, 王亚沙, 刘驰. 大数据技术前瞻[J]. 大数据, 2023, 9(1): 1-20.
[3]	沈阳, 余梦珑. 元宇宙与大数据：时空智能中的数据洞察与价值连接[J]. 大数据, 2023, 9(1): 103-110.
[4]	郑童哲恒, 李斌, 冯敏萱, 常博林, 王东波. 历史典籍的结构化探索——《史记·列传》数字人文知识库的构建与可视化研究[J]. 大数据, 2022, 8(6): 40-55.
[5]	陈静. 人文大数据及其在数字人文领域中的应用[J]. 大数据, 2022, 8(6): 3-14.
[6]	罗煜楚, 吴昊, 郭宇涵, 谭绍聪, 刘灿, 蒋瑞珂, 袁晓如. 数字人文中的可视化[J]. 大数据, 2022, 8(6): 74-93.
[7]	李汶龙, 袁媛, 安筱鹏. 刍议大数据治理的三大基础思维[J]. 大数据, 2022, 8(4): 34-45.
[8]	汤奇峰, 邵志清, 叶雅珍. 数据交易中的权利确认和授予体系[J]. 大数据, 2022, 8(3): 40-53.
[9]	王陈慧子, 蔡玮. 元宇宙数字经济：现状、特征与发展建议[J]. 大数据, 2022, 8(3): 140-150.
[10]	杨玫, 李玮, 乔思渊, 刘巍. 中国大数据产业产值测算方法研究[J]. 大数据, 2022, 8(3): 151-160.
[11]	李德仁, 张过, 蒋永华, 沈欣, 刘伟玲. 论大数据视角下的地球空间信息学的机遇与挑战[J]. 大数据, 2022, 8(2): 3-14.
[12]	仇晓兰, 胡玉新, 上官松涛, 付琨. 遥感卫星大数据高精度一体化处理技术[J]. 大数据, 2022, 8(2): 15-27.
[13]	刘伟权, 王程, 臧彧, 胡倩, 于尚书, 赖柏锜. 基于遥感大数据的信息提取技术综述[J]. 大数据, 2022, 8(2): 28-57.
[14]	刘建强, 叶小敏, 兰友国. 我国海洋卫星遥感大数据及其应用服务[J]. 大数据, 2022, 8(2): 75-88.
[15]	杨何群, 王晓峰, 高彦青, 陆一闻, 麻炳欣, 王昕瑶. 数值天气预报对卫星大数据的需求分析[J]. 大数据, 2022, 8(2): 89-102.