storm平台下工作节点的内存电压调控节能策略

doi:10.11959/j.issn.1000-436x.2018213

Abstract

Abstract:

Focused on the problem that traditional energy-efficient strategies never consider about the real time of data processing and transmission,models of directed acyclic graph,parallelism of instance,resource allocation for task and critical path were set up based on the features of data stream processing and the structure of storm cluster.Meanwhile,the WNDVR-storm (energy-efficient strategy for work node by dram voltage regulation in storm) was proposed according to the analysis of critical path and system performance,which included two energy-efficient algorithms aiming at whether there were any work nodes executing on the non-critical path of a topology.Finally,the appropriate threshold values fit for the CPU utilization of work node and the volume of transmitted data were determined based on the data processing and transmission constraints to dynamically regulate the DRAM voltage of the system.The experimental result shows that the strategy can reduce energy consumption effectively.Moreover,the fewer constraints are,the higher energy efficiency is.

Key words: big data, stream computing, storm, critical path, DRAM voltage, energy consumption

CLC Number:

TP311

Yonglin PU,Jiong YU,Liang LU,Chen BIAN,Bin LIAO,Ziyang LI. Energy-efficient strategy for work node by DRAM voltage regulation in storm[J]. Journal on Communications, 2018, 39(10): 97-117.

Figures/Tables 19

References 15

[16]	TCHEMYKH A , PECERO J E , BARRONDO A ,et al. Adaptive energy efficient scheduling in Peer-to-Peer desktop grids[J]. Future Generation Computer Systems, 2014,36(7): 209-220.
[17]	POLLAKIS E , CAVALCANTE R L G , STANCZAK S . Traffic demand-aware topology control for enhanced energy-efficiency of cellular networks[J]. EURASIP Journal on Wireless Communications and Networking, 2016,2016(1): 61-77.
[18]	LIU G X , XU J L , HONG X B . Internet of things sensor node information scheduling model and energy saving strategy[J]. Advanced Materials Research, 2013,773(1): 215-220.
[19]	WANG X , WANG Y , CUI Y . A new multi-objective bi-level programming model for energy and locality aware multi-job scheduling in cloud computing[J]. Future Generation Computer Systems, 2014,36(7): 91-101.
[20]	HE F Y , WANG F . Research on energy saving for multi-back after blending water from one station system’s parameters optimization[J]. Advanced Materials Research, 2014,1023: 187-191.
[21]	MILI M R , MUSAVIAN L , HAMDI K A ,et al. How to increase energy efficiency in cognitive radio networks[J]. IEEE Transactions on Communications, 2016,64(5): 1829-1843.
[22]	CHEN X , YANG H , ZHANG W . A comprehensive sensitivity study of major passive design parameters for the public rental housing development in Hong Kong[J]. Energy, 2015,(93): 1804-1818.
[23]	WANG H , CHEN Q . A semi-empirical model for studying the impact of thermal mass and cost-return analysis on mixed-mode ventilation in office buildings[J]. Energy ＆ Buildings, 2013,67(4): 267-274.
[24]	YANG T , MINO G , BAROLLI L ,et al. Energy-saving in wireless sensor networks considering mobile sensor nodes[J]. Computer Systems Science ＆ Engineering, 2011,27(5): 317-326.
[25]	孙大为, 张广艳, 郑纬民 . 大数据流式计算:关键技术及系统实例[J]. 软件学报, 2014,25(4): 839-862.
	SUN D W , ZHANG G Y , ZHENG W M . Big data stream computing:Technologies and instances[J]. Journal of Software, 2014,25(4): 839-862.
[26]	LI K C , JIANG H , YANG L T ,et al. Big data:algorithms,analytics,and applications[M]. Florida: CRC PressPress, 2015: 193-214.
[27]	BONAMY R , BILAVARN S , MULLER F . An energy-aware scheduler for dynamically reconfigurable multi-core systems[C]// International Symposium on Reconfigurable Communication-Centric Systems-On-Chip. 2015: 1-6.
[28]	蒲勇霖, 于炯, 鲁亮 ,等. 大数据流式计算环境下的内存节能策略[J]. 小型微型计算机系统, 2017,38(9): 1988-1993.
	PU Y L , YU J , LU L ,et al. Energy-efficient strategy for memory in big data stream computing environment[J]. Journal of Chinese Mini-Micro Computer Systems, 2017,38(9): 1988-1993.
[29]	TRIHINAS D , PALLIS G , DIKAIAKOS M D . JCatascopia:monitoring elastically adaptive applications in the cloud[C]// The 14th IEEE/ACM Int Symp on Cluster,Cloud and Grid Computing (CCGrid). 2014: 226-235.
[30]	VAN D V J S , VAN D W B , LAZOVIK E ,et al. dynamically scaling apache storm for the analysis of streaming data[C]// The 2015 IEEE 1st Int Conf on Big Data Computing Service and Applications. 2015: 154-161.
[31]	CORDESCHI N , SHOJAFAR M , AMENDOLA D ,et al. Energy-efficient adaptive networked datacenters for the QoS support of real-time applications[J]. The Journal of Supercomputing, 2014,71(2): 448-478.
[32]	BASKIYAR S , ABDEL-KADER R . Energy aware DAG scheduling on heterogeneous systems[J]. Cluster Computing, 2010,13(4): 373-383.
[33]	HSU C H , SLAGTER K D , CHEN S C ,et al. Optimizing energy consumption with task consolidation in clouds[J]. Information Sciences, 2014,258(3): 452-462.
[34]	KIM N , CHO J , SEO E . Energy-credit scheduler:an energy-aware virtual machine scheduler for cloud systems[J]. Future Generation Computer Systems, 2014,32(2): 128-137.
[35]	ZONG Z , MANZANARES A , RUAN X ,et al. EAD and PEBD:two energy-aware duplication scheduling algorithms for parallel tasks on homogeneous clusters[J]. IEEE Transactions on Computers, 2010,60(3): 360-374.
[36]	PATAN R , RAJASEKHARA B M . Re-storm:real-time energy efficient data analysis adapting storm platform[J]. Jurnal Teknologi, 2016,78(10): 139-146.
[37]	鲁亮, 于炯, 卞琛 ,等. 大数据流式计算框架Storm的任务迁移策略[J]. 计算机研究与发展, 2018,55(1): 71-92.
	LU L , YU J , BIAN C ,et al. A Task Migration Strategy in Big DataStream Computing with Storm[J]. Journal of Computer Research and Development, 2018,55(1): 71-92.
[38]	MATTEIS T D , MENCAGLI G . Keep calm and react with foresight:strategies for low-latency and energy-efficient elastic data stream processing[J]. Journal of Systems and Software, 2016,51(8): 1-12.
[39]	PAMLEY M R , ORRO J M , KEOWN W F ,et al. Dynamic memory voltage scaling for power management:US,US20100250981[P]. 2010.
[40]	蒲勇霖, 于炯, 王跃飞 ,等. 大数据流式计算环境下的阈值调控节能策略[J]. 计算机应用, 2017,37(6): 1580-1586.
	PU Y L , YU J , WANG Y F ,et al. Energy-efficient strategy for threshold control in big data stream computing environment[J]. Journal of Computer Applications, 2017,37(6): 1580-1586.
[41]	HONG S P , YOO S J . Dynamic voltage scaling method of CPU using workload estimator and computer readable medium storing the method:US,US7685446[P]. 2010.
[42]	HWANG I , PEDRAM M . A comparative study of the effectiveness of CPU consolidation versus dynamic voltage and frequency scaling in a virtualized multicore server[J]. IEEE Transactions on Very Large Scale Integration Systems, 2016,24(6): 2103-2116.
[43]	林闯, 田源, 姚敏 . 绿色网络和绿色评价:节能机制、模型和评价[J]. 计算机学报, 2011,34(4): 593-612.
	LIN C , TIAN Y , YAO M . Green network and green evaluation:mechanism,modeling and evaluation[J]. Chinese Journal of Computers, 2011,34(4): 593-612.
[44]	GRIFFITHS N . Nmon performance:A free tool to analyze AIX and Linux performance[EB/OL].(2017-06-25)[2017-07-03]. .
[1]	孟小峰, 慈祥 . 大数据管理:概念、技术与挑战[J]. 计算机研究与发展, 2013,50(1): 146-169.
	MENG X F , CI X . Big data management:concepts,techniques and challenges[J]. Journal of Computer Research and Development, 2013,50(1): 146-169.
[2]	孙大为 . 大数据流式计算:应用特征和技术挑战[J]. 大数据, 2015,1(3): 99-105.
	SUN D W . Big data stream comuting:features and challenges[J]. Big Data Research, 2015,1(3): 99-105.
[3]	CHEN C L P , ZHANG C Y . Data-intensive applications,challenges,techniques and technologies:a survey on big data[J]. Information Sciences, 2014,275(11): 314-347.
[4]	RANJAN R . Streaming big data processing in datacenter clouds[J]. IEEE Cloud Computing, 2014,1(1): 78-83.
[5]	KAMBATLA K , KOLLIAS G , KUMAR V ,et al. Trends in big data analytics[J]. Journal of Parallel and Distributed Computing, 2014,74(7): 2561-2573.
[6]	邓维, 刘方明, 金海 ,等. 云计算数据中心的新能源应用:研究现状与趋势[J]. 计算机学报, 2013,36(3): 582-598.
	DENG W , LIU F M , JIN H ,et al. Leveraging renewable energy in cloud computing datacenters:state of the art and future research[J]. Chinese Journal of Computers, 2013,36(3): 582-598.
[7]	廖彬, 张陶, 于炯 ,等. 温度感知的 MapReduce 节能任务调度策略[J]. 通信学报, 2016,37(1): 61-75.
	LIAO B , ZHANG T , YU J ,et al. Temperature aware energy-efficient task scheduling strategies for MapReduce[J]. Journal on Communications, 2016,37(1): 61-75.
[8]	蒲勇霖, 于炯, 鲁亮 ,等. 基于实时流式计算系统的数据分类节能策略[J]. 计算机工程与设计, 2017,38(1): 59-64.
	PU Y L , YU J , LU L ,et al. Energy-efficient strategy based on data classification in real-time stream computing system[J]. Computer Engineering and Design, 2017,38(1): 59-64
[9]	SUN D , ZHANG G , YANG S ,et al. Re-Stream:Real-time and energy-efficient resource scheduling in big data stream computing environments[J]. Information Sciences, 2015,(319): 92-112.
[10]	TOSHNIWAL A , TANEJA S , SHUKLA A ,et al. Storm @Twitter[C]// Proc of the 2014 ACM SIGMOD Int Conf on Management of data. ACM, 2014: 147-156.
[11]	BORTHAKUR D , GRAY J , SARMA J S ,et al. Apache hadoop goes realtime at Facebook[C]// The 2011 ACM SIGMOD Int Conf on Management of data. 2011: 1071-1080.
[12]	NEUMEVER L , ROBBINS B , NAIR A ,et al. S4:Distributed stream computing platform[C]// The 10th IEEE Int Conf on Data Mining Workshops (ICDMW 2010). 2010: 170-177.
[13]	ZAHARIA M , DAS T , LI H ,et al. Discretized streams:an efficient and fault-tolerant model for stream processing on large clusters[C]// The 4th USENIX conf on Hot Topics in Cloud Computing. 2012:10.
[14]	FISCHER M J , SU X , YIN Y . Assigning tasks for efficiency in hadoop[C]// The 22th annual ACM Symp on Parallelism in algorithms and architectures. 2010: 30-39.
[15]	ALBERS S . Energy-efficient algorithms[J]. Communications of the ACM, 2010,53(5): 86-96.

Metrics

Recommended 0

No Suggested Reading articles found!

节点	CPU	内存	硬盘
nimbus	Intel core i7	4 GB DDR3	1 TB
zookeeper	4790 3.6 GHz	1 066 MHz
1(leader)	Quad Core	（常电压）
supervisor	Intel core i7	4 GB DDR3	1 TB
1~16	4790 3.6 GHz	1 066 MHz
	Quad Core	（常电压）
zookeeper	Intel core i7	4 GB DDR3	1 TB
2、3	4790 3.6 GHz	1 066 MHz
(follower)	Quad Core	（常电压）

参数	数值
操作系统	CentOS 6.4
内存电压/V	1.50
storm版本	0.9.6
Java运行环境	1.7 Open JDK
network	100 Mbit/s LAN
zookeeper	3.4.5

基准测试	参数	值
	component.spout_num	16
	component.split_bolt_num	32
WordCount	component.count_bolt_num	32
	topology.works	16
	Topology.acker.executors	16
	topology.max.spout.pending	200
	component.spout_num	16
	component.sort_bolt_num	32
	emit.frequency	300
RollingSort	chunk.size	2 000 000
	message.size	10 000
	topology.works	16
	Topology.acker.executors	16
	topology.max.spout.pending	200
	topology.level	3
	message.size	100
Sol	component.spout_num	16
	component.bolt_num	32
	topology.works	16
	Topology.acker.executors	16
	topology.max.spout.pending	200
	component.spout_num	16
	component.split_bolt_num	32
	component.rolling_count_bolt_num	32
RollingCount	window.length	150
	emit.frequency	30
	topology.works	16
	Topology.acker.executors	16
	topology.max.spout.pending	200

状态	参数/kJ
空闲状态	14.32
活动状态	27.03
休眠状态	5.12
空闲同步	21.86
休眠同步	15.57
I/O读状态	16.72
I/O写状态	24.61

条件	最小功率/W	最大功率/W	平均功率/W
test11	660.599 8	1 249.700 64	1 031.364 624
test12	659.559 76	1 133.421 38	944.682 701 6
test13	620.458 24	1 091.645 24	876.792 827 5
test14	682.106 44	1 220.386	968.984 928 2
test21	660.599 8	1 249.700 64	1 031.364 624
test22	659.523 2	1 097.862 18	922.752 307 5
test23	621.864 12	966.097 32	782.294 926 2
test24	641.513 26	1 044.151 48	849.543 725 6
test31	649.030 2	1 280.407 04	1 122.897 352
test32	630.920 3	1 133.241 46	1 001.489 621
test33	620.041 62	1 030.159 06	889.198 378
test34	601.059 78	1 185.903 68	1 017.047 417
test41	649.030 2	1 280.407 04	1 122.897 352
test42	603.998 78	1 097.230 56	978.604 427 5
test43	657.325 69	884.586 23	773.658 649 5
test44	642.205 18	1 090.140 84	968.210 853 8
test51	638.219 76	1 262.303 82	1 027.447 87
test52	638.939	1 202.766 48	926.434 571 1
test53	663.415 83	1 117.752 49	859.454 434 8
test54	619.521 5	1 150.994 39	923.691 601 6
test61	638.219 76	1 262.303 82	1 027.447 87
test62	625.125 14	1 003.757 6	858.255 253 8
test63	626.493 78	917.566 48	769.225 966 9
test64	645.524 6	1 024.253 72	923.184 047 4

Energy-efficient strategy for work node by DRAM voltage regulation in storm

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 19

References 15

Related Articles 15

Metrics

Recommended 0

[1]	Weifeng LU, Ning LI, Jia XU, Lijie XU, Jian XU. Joint scheduling algorithm for correlative tasks in multi-access edge computing [J]. Journal on Communications, 2023, 44(4): 87-98.
[2]	Xueyong YU, Lixiang QIU, Jianing SONG, Hongbo ZHU. Security communication and energy efficiency optimization strategy in UAV-aided edge computing [J]. Journal on Communications, 2023, 44(3): 45-54.
[3]	Wei JIN, Fenghua LI, Mingjie YU, Yunchuan GUO, Ziyan ZHOU, Liang FANG. HDFS-oriented cryptographic key resource control mechanism [J]. Journal on Communications, 2022, 43(9): 27-41.
[4]	Cuiran LI, Xuejie WANG, Jianli XIE, Anqi LYU. Routing algorithm for railway monitoring linear WSN based on improved PSO [J]. Journal on Communications, 2022, 43(5): 155-165.
[5]	Yimin MAO, Qianhu DENG, Zhigang CHEN. Parallel association rules incremental mining algorithm based on information entropy and genetic algorithm [J]. Journal on Communications, 2021, 42(5): 122-136.
[6]	Liang YUAN, Xiao YU, Enjie DING, Xiaohu ZHAO, Shimin FENG, Da ZHANG, Tongyu LIU, Weidong WANG, Yanqiu HUANG. Research on key technologies of human-machine-environment states perception in mine Internet of things [J]. Journal on Communications, 2020, 41(2): 1-12.
[7]	Ziyang LI,Jiong YU,Yuefei WANG,Chen BIAN,Yonglin PU,Yitian ZHANG,Yu LIU. Load prediction based elastic resource scheduling strategy in Flink [J]. Journal on Communications, 2020, 41(10): 92-108.
[8]	. Dual-architecture Internet supporting intelligent governance of cyber content [J]. Journal on Communications, 2019, 40(9): 1-14.
[9]	Ziyang LI,Jiong YU,Chen BIAN,Yitian ZHANG,Yonglin PU,Yuefei WANG,Liang LU. Flow-network based auto rescale strategy for Flink [J]. Journal on Communications, 2019, 40(8): 85-101.
[10]	Yonglin PU,Jiong YU,Liang LU,Ziyang LI,Chen BIAN,Bin LIAO. Energy-efficient strategy for data migration and merging in Storm [J]. Journal on Communications, 2019, 40(12): 68-85.
[11]	Yu FU, Yihan YU, Xiaoping WU. Differential privacy protection technology and its application in big data environment [J]. Journal on Communications, 2019, 40(10): 157-168.
[12]	Wei LIU,Shu XIONG,Wei DU,Wei WANG. Research on Cloudlet selection strategy for data streaming applications in mobile cloud environment [J]. Journal on Communications, 2019, 40(1): 87-101.
[13]	Kun-fang ZHANG,Ming-ming LU,Lin ZHENG. Big data based metro crowd delivery system [J]. Journal on Communications, 2017, 38(Z2): 99-112.
[14]	Le-tian SHA,Fu XIAO,Wei CHEN,Jing SUN,Ru-chuan WANG. Sensitive information leakage awareness method for big data platform based on multi-attributes decision-making and taint tracking [J]. Journal on Communications, 2017, 38(7): 56-69.
[15]	Min NIE,Li SHI,Guang YANG,Chang-xing PEI. Influence of thunderstorm cloud on the performance of satellite-to-ground quantum link and parameters simulation [J]. Journal on Communications, 2017, 38(5): 31-38.