以实际算法为例评估MapReduce在石油勘探中的应用

doi:10.3969/j.issn.1000-436x.2012.z2.010

Abstract

Abstract:

Huge amounts of seismic data undergo complex iterative processing in the oil industry to get knowledge of the earth’s subsurface structure to detect where oil can be found and recovered.To evaluate the suitability of MapReduce for seismic processing algorithms,the algorithm design and implementation of Fresnel tomography on Hadoop MapReduce was described.Experiments demonstrate that MapReduce is approximately 3 times slower than MPI,and tuning the performance of MapReduce is really hard.To expand its applicability to high performance computing for oil industry,MapReduce should be improved in the flexibility and provide the opportunity to exploit fine-grained thread-level parallelism.Finally,research ideas to achieve these objectives were presented.

Key words: oil exploration, MapReduce, Fresnel tomography, high performance computing, seismic data

Chang-hai ZHAO,Hai-hua YAN,Xiao-peng LIU,Deng XIONG,Xiao-hua SHI. Evaluating MapReduce for seismic data processing using a practical application[J]. Journal on Communications, 2012, 33(Z2): 81-89.

Figures/Tables 8

References 30

[14]	The apache software foundation. MapReduce tutorial[EB/OL]. , 2012.
[15]	王珊, 王会举, 覃雄派 ,等. 架构大数据:挑战、现状与展望[J]. 计算机学报, 2011,34(10): 1741-1752. WANG S , WANG H J , TAN X P ,et al. Architecting big data:challenges,studies and forecasts[J]. Chinese Journal of Computers, 2011,34(10): 1741-1752.
[16]	RAICU I , FOSTER I T , BECKMAN P . Making a case for distributed file systems at Exascale[A]. LSAP '11[C]. San Jose,USA, 2011. 11-18.
[17]	YANG H , DASDAN A , HSIAO R ,et al. Map-reduce-merge:simplified relational data processing on large clusters[A]. SIGMOD '07[C]. Beijing,China, 2007. 1029-1040.
[18]	EKANAYAKE J , LI H , ZHANG B ,et al. Twister:a runtime for iterative MapReduce[A]. HPDC '10[C]. Chicago,USA, 2010. 810-818.
[19]	ZHANG B , RUAN Y , WU T ,et al. Applying twister to scientific applications[A]. CLOUDCOM'10[C]. Indianapdis,USA, 2010. 25-32.
[20]	KAMBATLA K , PATHAK A , PUCHA H . Towards optimizing hadoop provisioning in the cloud[A]. HotCloud'09[C]. San Diego,CA,USA, 2009.
[21]	SANDHOLM T , LAI K . MapReduce optimization using regulated dynamic prioritization[A]. SIGMETRICS'09[C]. 2009. 299-310.
[22]	ZAHARIA M , Konwinski A , Joseph A D ,et al. Improving MapReduce performance in heterogeneous environments[A]. OSDI'08[C]. San Diego,CA,USA, 2008. 29-42.
[23]	EKANAYAKE J , PALLICKARA S , FOX G . Mapreduce for data intensive scientific analyses[A]. IEEE Fourth International Conference on eScience[C]. Indianapdis,USA, 2008. 277-284.
[24]	TALBOT J , YOO R M , KOZYRAKIS C . Phoenix++:modular MapReduce for shared-memory systems[A]. MapReduce '11[C].San Jose,USA. 2011. 9-16.
[25]	RANGER C , RAGHURAMAN R , PENMETSA A ,et al. Evaluating Mapreduce for multi-core and multiprocessor systems[A]. IEEE 13th International Symposium on High Performance Computer Architecture(HPCA)[C]. Phoenix,USA, 2007. 13-24.
[26]	JIANG W , RAVI V T , AGRAWAL G . A Map-Reduce system with an alternate API for multi-core environments[A]. CCGRID '10[C]. Melbourne,Australia, 2010. 84-93.
[27]	HE B , FANG W , LUO Q ,et al. Mars:a MapReduce framework on graphics processors[A]. PACT'08[C]. Toronto,Canada, 2008. 260-269.
[1]	赵改善 . 我们需要多大和多快的计算机[J]. 勘探地球物理进展, 2004,27(1): 22-28. ZHAO G S . How big and fast computers can meet out needs[J]. Progress in Exploration Geophysics, 2004,27(1): 22-28.
[2]	赵改善 . 高性能计算在石油物探中的应用现状与前景[J]. 高性能计算发展与应用, 2009(29): 19-23. ZHAO G S . Current status and outlook of high performance computing for seismic exploration[J]. Development and Application of High Performance Computing, 2009(29): 19-23.
[28]	CHANG H , VANDYKE J P , SOLANO M ,et al. 3-D prestack kirchhoff depth migration:from prototype to production in a massively parallel processor environment[J]. Geophysics, 1998,63(2): 546-556.
[29]	SAULE E , CATALYUREK U V . An early evaluation of the scalability of graph algorithms on the Intel MIC architecture[A]. 26th International Symposium on Parallel and Distributed Processing,Workshops and PhD Forum (IPDPSW),Workshop on Multithreaded Architectures and Applications(MTAAP)[C]. Shanghai,China, 2012.
[3]	BAUTISTA-GOMEZ L , TSUBOI S , KOMATITSCH D ,et al. FTI:high performance fault tolerance interface for hybrid systems[A]. SC '11[C]. Seattle,WA,USA, 2011. 31-32.
[4]	DEAN J , GHEMAWAT S . MapReduce:simplified data processing on large clusters[J]. Communications of the ACM, 2008,51(1): 107-113.
[5]	Oracle Inc . Lustre:a scalable high-performance file system[EB/OL]. , 2012.
[6]	SCHMUCK F , HASKIN R . GPFS:a shared-disk file system for large computing clusters[A]. FAST'02[C]. Monterey,CA,USA, 2002. 231-244.
[7]	SRIRAMA S N , JAKOVITS P , VAINIKKO E . Adapting scientific computing problems to clouds using MapReduce[J]. Future Generation Computer Systems, 2012,28(1): 184-192.
[8]	FADIKA Z , DEDE E , GOVINDARAJU M ,et al. MARIANE:MApReduce implementation adapted for HPC environments[A]. GRID'11[C]. Lyon,France, 2011. 82-89.
[9]	GUNARATHNE T , WU T , QIU J ,et al. MapReduce in the clouds for science[A]. CLOUDCOM'10[C]. USA, 2010. 565-572.
[10]	MACKEY G , SEHRISH S , BENT J ,et al. Introducing Map-reduce to high end computing[A]. Petascale Data Storage Workshop,2008.PDSW'08[C]. Austin,USA, 2008. 1-6.
[11]	Committee of SEG Technical Standards . SEG Y rev1 Data Exchange format[S]. Society of Exploration of Geophysicist, 2002.
[12]	张建中, 杨国辉, 林文 ,等. Fresnel层析成像并行算法研究[J]. 计算机研究与发展. 2007,44(10): 1661-1666. ZHANG J Z , YANG G H , LIN W ,et al. A parallel algorithm for fresnel tomography[J]. Journal of Computer Research and Development, 2007,44(10): 1661-1666.
[13]	刘玉柱, 董良国, 李培明 ,等. 初至波菲涅尔体地震层析成像[J]. 地球物理学报. 2009,52(9): 2310-2320. LIU Y Z , DONG L G , LI P M ,et al. Fresnel volume tomography based on the first arrival of the seismic wave[J]. Chinese Journal of Geophysics, 2009,52(9): 2310-2320.
[30]	Intel many integrated core architecture[EB/OL]. , 2012.

Metrics

Recommended 0

No Suggested Reading articles found!

系统	配置
nodes	256
CPUs in each node	2,Intel? Xeon? Processor X5650 2.67GHz
Threads in each CPU	6(Cores)× 2(Threads)
Memory Per Node	24GB
Network	Gigabit
Disks	1,SCSI disk,Read Speed:171MB/sec
I/O servers	8
Software	Hadoop 0.21.0,Sun JDK 6,MPICH2 1.2
OS	Red Hat Enterprise Linux Server release 5.4-64 bit

Evaluating MapReduce for seismic data processing using a practical application

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 30

Related Articles 12

Metrics

Recommended 0

[1]	Yuanzhao GAO,Binglong LI,Xingyuan CHEN. Stochastic algorithm for HDFS data theft detection based on MapReduce [J]. Journal on Communications, 2018, 39(10): 11-21.
[2]	Yihan YU,Yu FU,Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework [J]. Journal on Communications, 2018, 39(1): 70-77.
[3]	Hong-cheng LI,Xiao-ping WU,Yan CHEN. k-means clustering method preserving differential privacy in MapReduce framework [J]. Journal on Communications, 2016, 37(2): 125-131.
[4]	Hai-yan WANG,Pan CAO. Information extraction from massive Web pages based on node property and text content [J]. Journal on Communications, 2016, 37(10): 9-17.
[5]	Bin LIAO,Tao ZHANG,Jiong YU,Ji LIU,tong YINLu,Gang GUO. Temperature aware energy-efficient task scheduling strategies for mapreduce [J]. Journal on Communications, 2016, 37(1): 61-75.
[6]	. Research on distributed genetic k-means for anomaly detection in MANET [J]. Journal on Communications, 2015, 36(11): 167-173.
[7]	Qing-ni SHEN,Si-han QING,Zhong-hai WU,Li-zhe ZHANG,Ya-hui YANG. Securely redundant scheduling policy for MapReduce based on dynamic domains partition [J]. Journal on Communications, 2014, 35(1): 34-46.
[8]	. Design and application research on data service platform for big data [J]. Journal on Communications, 2013, 34(Z2): 32-174.
[9]	Yun-feng LIU,Li LI,Su-mei WANG,Qian-yi WANG,Xu YANG,Rong-bin OUYANG,Xin-zheng LONG. Design and application research on data service platform for big data [J]. Journal on Communications, 2013, 34(Z2): 170-174.
[10]	Chun-ling CHENG,Qian CHAI,Xiao-long XU. Cloud computing virus detection algorithm based on co-evolution immune networ k [J]. Journal on Communications, 2011, 32(9A): 1-7.
[11]	Zhong-gang WANG,Tian-rui LI,Jun-bo ZHANG,Cheng-bing ZHAO,Zi-zhe GAO. Parallel method for preprocessing high-speed railway noise data based on MapReduce [J]. Journal on Communications, 2011, 32(9A): 263-269.
[12]	Jie XIN,Zhi-ming CUI,Peng-peng ZHAO,Guang-ming ZHANG,Xue-feng XIAN. Applying MapReduce frameworks to a virtualization platform for Deep Web data source discovery [J]. Journal on Communications, 2011, 32(7): 189-195.