大数据场景中语言虚拟机的应用和挑战

doi:10.11959/j.issn.2096-0271.2020035

Abstract

Abstract:

Language virtual machines provide a platform-independent execution environment for big-data applications and simplify their development and deployment phases,so they are widely used in the big-data scenario.The applications of two different kinds of mainstream language virtual machines:JVM and CLR,were analyzed,and four challenges when adopting language virtual machines:initialization and warm-up overhead,garbage collection pauses,heterogeneous memory support,and data layout transformation,were summarized.Afterward,existing approaches to the challenges were discussed and their shortcomings and possible optimizations in the future were analyzed.

Key words: language virtual machines, garbage collection, heterogeneous memory

CLC Number:

TP315

Mingyu WU, Haibo CHEN, Binyu ZANG. Applications and challenges of language virtual machines in big data[J]. Big Data Research, 2020, 6(4): 81-91.

Figures/Tables 1

References 37

[1]	WHITE T . Hadoop:the definitive guide[M]. Sebastopol: O’Reilly Media,Inc.Press, 2012:647.
[2]	ZAHARIA M , XIN R , WENDELL P ,et al. Apache Spark:a unified engine for big data processing[J]. Communications of the ACM, 2016,59(11): 56-65.
[3]	CARBONE P , KATSIFODIMOS A , EWEN S ,et al. Apache Flink:stream and batch processing in a single engine[J]. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015,36(4): 28-38.
[4]	YU Y , ISARD M , FETTERLY D ,et al. DryadLINQ:a system for general-purpose distributed data-parallel computing using a high-level language[C]// The 8th USENIX Symposium on Operating Systems Designand Implementation. Berkeley:USENIX Association, 2008: 383-400.
[5]	LINDHOLM T , YELLIN F , BRACHA G ,et al. The Java virtual machine specification[M]. London: Pearson EducationPress, 2014:584.
[6]	KENNEDY A , SYME D.Design and implementation of generics for the . NET common language runtime[C]// The ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation. New York:ACM Press, 2001: 1-12.
[7]	DU D Y . Apache Hive essentials[M]. Birmingham: Packt Publishing LtdPress, 2015:191.
[8]	BORKAR V , CAREY M , GROVER R ,et al. Hyracks:a flexible and extensible foundation for data-intensive computing[C]// The IEEE 27th International Conference on Data Engineering. Piscataway:IEEE Press, 2011: 1151-1162.
[9]	IQBAL M H , SOOMRO T R . Big data analysis:Apache Storm perspective[J]. International Journal of Computer Trends and Technology, 2015,19(1): 9-14.
[10]	ISARD M , BUDIU M , YU Y ,et al. Dryad:distributed data-parallel programs from sequential building blocks[C]// The 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems. New York:ACM Press, 2014: 59-72.
[11]	CHAIKEN R , JENKINS B , LARSON P ,et al. SCOPE:easy and efficient parallel processing of massive data sets[J]. Proceedings of the VLDB Endowment, 2008,1(2): 1265-1276.
[12]	PIALORSI P , RUSSO M . Introducing microsoft? LINQ[M]. Redmond: Microsoft PressPress, 2007:282.
[13]	LION D , CHIU A , SUN H ,et al. Don’t get caught in the cold,warm-up your JVM:understand and eliminate JVM warm-up overhead in data-parallel systems[C]// The 12th USENIX Symposium on Operating Systems Design and Implementation. Berkeley:USENIX Association, 2016: 383-400.
[14]	UNGAR D , . Generation scavenging:a nondisruptive high performance storage reclamation algorithm[C]// The 1st ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments. New York:ACM Press, 1984: 157-167.
[15]	NGUYEN K , FANG L , XU G Q ,et al. Yak:a high-performance big-data-friendly garbage collector[C]// The 12th USENIX Symposium on Operating Systems Design and Implementation. Berkeley:USENIX Association, 2016: 349-365.
[16]	OpenJDK. JEP 316:heap allocation on alternative memory devices[Z]. 2016.
[17]	DRABAS T , LEE D . Learning PySpark[M]. Birmingham: Packt Publishing LtdPress, 2017:250.
[18]	VENKATARAMAN S , YANG Z H , LIU D ,et al. SparkR:scaling R programs with Spark[C]// The 2016 International Conference on Management of Data. New York:ACM Press, 2016: 1099-1104.
[19]	NGUYEN K , FANG L , NAVASCA C ,et al. Skyway:connecting managed heaps in distributed big data systems[C]// The 23rd International Conference on Architectural Support for Programming Languages and Operating Systems. New York:ACM Press, 2018: 56-69.
[20]	OpenJDK. JEP 310:application class-data sharing[Z]. 2017.
[21]	PIRVU M . Optimize JVM start-up with Eclipse OpenJ9[R]. 2018.
[22]	WANG K A , HO R , WU P . Replayable execution optimized for page sharing for a managed runtime environment[C]// The 14th EuroSys Conference 2019. New York:ACM Press, 2019: 1-16.
[23]	BRUNO R , OLIVEIRA L P , FERREIRA P . NG2C:pretenuring garbage collection with dynamic generations for HotSpot big data applications[C]// The 2017 ACM SIGPLAN International Symposium on Memory Management. New York:ACM Press, 2017: 2-13.
[24]	NGUYEN K , WANG K , BU Y Y ,et al. FA?ADE:a compiler and runtime for (almost) object-bounded big data applications[C]// The 20th International Conference on Architectural Support for Programming Languages and Operating Systems. New York:ACM Press, 2015: 675-690.
[25]	XIN R , ROSEN J . Project Tungsten:bringing Apache Spark closer to BareMetal[R]. 2015.
[26]	SHI X H , KE Z X , ZHOU Y L ,et al. Deca:a garbage collection optimizer for in-memory data processing[J]. ACM Transactions on Computer Systems, 2019,36(1): 1-47.
[27]	FANG L , NGUYEN K , XU G Q ,et al. Interruptible tasks:treating memory pressure as interrupts for highly scalable data-parallel programs[C]// The 25th Symposium on Operating Systems Principles. New York:ACM Press, 2015: 394-409.
[28]	TAN J , CHIN A , HU Z Z ,et al. DynMR:dynamic MapReduce with reducetask interleaving and maptask backfilling[C]// The 9th European Conference on Computer Systems. New York:ACM Press, 2014: 1-14.
[29]	YANG J , KIM J , HOSEINZADEH M ,et al. An empirical guide to the behavior and use of scalable persistent memory[C]// The 18th USENIX Conference on File and Storage Technologies. Berkeley:USENIX Association, 2020: 169-182.
[30]	WANG C X , CUI H M , CAO T ,et al. Panthera:holistic memory management for big data processing over hybrid memories[C]// The 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. New York:ACM Press, 2019: 347-362.
[31]	WU M Y , CHEN H B , ZHU H ,et al. GCPersist:an efficient GC-assisted lazy persistency framework for resilient Java applications on NVM[C]// The 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. New York:ACM Press, 2020: 1-14.
[32]	XU G Q , VEANES M , BARNETT M ,et al. Niijima:sound and automated computation consolidation for efficient multilingual data-parallel pipelines[C]// The 27th ACM Symposium on Operating Systems Principles. New York:ACM Press, 2019: 306-321.
[33]	NAVASCA C , CAI C , NGUYEN K ,et al. Gerenuk:thin computation over big native data using speculative program transformation[C]// The 27th ACM Symposium on Operating Systems Principles. New York:ACM Press, 2019: 538-553.
[34]	KRYO. Java binary serialization and cloning:fast,efficient,automatic[R]. 2020.
[35]	COLFER. The Colfer serializer[R]. 2017.
[36]	OpenJDK.Valhalla[R]. 2020.
[37]	GraalVM.Run programs faster anywhere[R]. 2020.

Metrics

Recommended 0

No Suggested Reading articles found!

Applications and challenges of language virtual machines in big data

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 1

References 37

Related Articles 14

Metrics

Recommended 0

[1]	Peiwu CHEN, Fangxing SHU. A recommender algorithm based on SVD ++model under trust network [J]. Big Data Research, 2021, 7(4): 105-116.
[2]	Zhao ZHANG, Jixin TIAN, Cheqing JIN. On-chain witness and off-chain transmission trustworthy data sharing platform [J]. Big Data Research, 2020, 6(5): 106-117.
[3]	Jian YIN, Huaijie ZHU, Jianxing YU, Shuang QIU. A panoramic framework of big data governance [J]. Big Data Research, 2020, 6(2): 19-26.
[4]	. Sharing,integration and fusion of governmentgovernance big data [J]. Big Data Research, 2020, 6(2): 27-40.
[5]	Wanggen LIU, Huaicheng ZHENG, Guoping RONG. A scheduler system for large-scale distributed data computing in cloud [J]. Big Data Research, 2020, 6(1): 81-98.
[6]	Minghe YU, Tiezheng NIE, Guoliang LI. Data curation technologies and applications [J]. Big Data Research, 2019, 5(6): 30-46.
[7]	Ying LIANG, Wei ZHANG, Zhidong YU, Hongzhou SHI. Applications of academic big data in the process of science and technology management [J]. Big Data Research, 2019, 5(5): 3-15.
[8]	Xiaoyong DU, Yueguo CHEN, Ju FAN, Wei LU. Data wrangling:a key technique of data governance [J]. Big Data Research, 2019, 5(3): 13-22.
[9]	Lei LIU, Zhijun GUO, Haixin MA, Qiong ZHAO, Huiqi HU, Peng CAI, Hongtao DU, Aoying ZHOU, Zhanhuai LI. Exploration and applications of distributed database in financial area [J]. Big Data Research, 2019, 5(1): 77-86.
[10]	HEN Shimin C. Efficient support of tree-structured data types [J]. Big Data Research, 2018, 4(4): 35-43.
[11]	Youhuan LI, Lei ZOU. Graph stream:model,algorithm and system [J]. Big Data Research, 2018, 4(4): 44-55.
[12]	Shu YAN, Sude QING, Kai WEI. Application of blockchain in data circulation [J]. Big Data Research, 2018, 4(1): 3-12.
[13]	Weining QIAN, Cheqing JIN, Qifeng SHAO, Aoying ZHOU. Blockchain and sharing database [J]. Big Data Research, 2018, 4(1): 36-45.
[14]	. Brief comments on the operability of blockchain [J]. Big Data Research, 2018, 4(1): 85-89.