面向多星多任务的大数据处理系统设计

doi:10.11959/j.issn.2096-0271.2021045

Abstract

Abstract:

With more and more space science satellites were launched in China, the scientific data amount has grown explosively, and the resulting space science satellite big data processing has gradually become a key link in the development of space science innovation.A high-performance ground data processing system is an important driver for promoting the construction of Chinese controllable space science big data ecology and boosting the output of scientific results.Aiming at the characteristics of multi-level classification, multi-source product integration organization and high timeliness requirements in space science satellite data processing, and for the task requirements of multi-satellite and multi-task parallel processing, a high-reliability hardware system design suitable for big data processing business scenarios was proposed, and a unified resource scheduling system based on task type perception was proposed according to the characteristics of scientific satellite processing tasks.Based on these designs, the development of a space science big data scalable ground big data processing system was completed, which had successfully supported the data processing tasks for space science missions from Chinese Academy of Sciences.

Key words: scientific satellite, big data, scalable, ground processing system

CLC Number:

V557

Fuli MA, Tao SHI, Ling CHEN, Yan ZHENG, Senlin XIONG. Design of big data processing system supporting multi-satellites and multi-tasks[J]. Big Data Research, 2021, 7(5): 3-16.

Figures/Tables 8

References 13

[1]	陈世荣, 严立, 申佩佩 . 高分遥感卫星影像预处理系统的设计和实现[J]. 测绘与空间地理信息, 2021,44(3): 1-3.
	CHEN S R , YAN L , SHEN P P . Design and implements of high-resolution satellite image preprocessing system[J]. Geomatics ＆Spatial Information Technology, 2021,44(3): 1-3.
[2]	刘定生, 陈元伟, 李景山 . 遥感卫星地面预处理系统技术发展模式探讨[J]. 遥感信息, 2008,23(5): 87-91.
	LIU D S , CHEN Y W , LI J S . The study on the development patterns of remote sensing satellite ground pre-processing systems[J]. Remote Sensing Information, 2008,23(5): 87-91.
[3]	MOSES J F , . Architectures toward reusable science data systems[C]// Proceedings of the AGU Fall Meeting.[S.l:s.n.], 2014.
[4]	钱建梅, 孙安来, 徐喆 ,等. 风云气象卫星数据存档与服务系统[J]. 应用气象学报, 2012,23(3): 369-376.
	QIAN J M , SUN A L , XU Z ,et al. Fengyun series meteorological satellite data archiving and service system[J]. Journal of Applied Meteorological Science, 2012,23(3): 369-376.
[5]	赵现纲, 林曼筠, 谢利子 ,等. 风云卫星地面应用系统计算机网络平台架构[J]. 气象科技进展, 2020,10(3): 43-48.
	ZHAO X G , LIN M J , XIE L Z ,et al. Computer network system of FY satellite ground segment[J]. Advances in Meteorological Science and Technology, 2020,10(3): 43-48.
[6]	李盛阳, 刘志文, 张万峰 ,等. 天宫二号应用任务地面数据处理与服务系统[J]. 中国科学:技术科学, 2020,50(8): 1066-1080.
	LI S Y , LIU Z W , ZHANG W F ,et al. Tiangong-2 application mission ground data processing and service system[J]. Scientia Sinica Technologica, 2020,50(8): 1066-1080.
[7]	BENKHOFF J , CASTEREN J V , HAYAKAWA H ,et al. BepiColombocomprehensive exploration of mercury:mission overview and science goals[J]. Planetary and Space Science, 2010,58(1-2): 2-20.
[8]	D’AMORE M , J?RN H , MATURILLI A ,et al. BepiColombo/MERTIS data processing pipeline at DLR[C]// Proceedings of the EGU General Assembly 2016.[S.l:s.n.], 2016.
[9]	DAMPE C . Direct detection of a break in the teraelectronvolt cosmic-ray spectrum of electrons and positrons[J]. Nature, 2017,552(7683): 63-66.
[10]	LIAO S K , CAI W Q , LIU W Y ,et al. Satelliteto-ground quantum key distribution[J]. Nature, 2017,549(7670): 43-47.
[11]	LIAO S K , YONG H L , LIU C ,et al. Long-distance free-space quantum key distribution in daylight towards intersatellite communication[J]. Nature Photonics, 2017,11(8): 509-513.
[12]	YIN J , CAO Y , LI Y H ,et al. Satellitebased entanglement distribution over 1200 kilometers[J]. Science, 2017,356(6343): 1140-1144.
[13]	LI T P , XIONG S L , ZHANG S N ,et al. Insight-HXMT observations of the first binary neutron star merger GW170817[J]. Science China Physics,Mechanics ＆Astronomy, 2017,61(3): 1-8.

Metrics

Recommended 0

No Suggested Reading articles found!

[1]	Haihong QIAN, Maoyi WANG, Yun XIONG. Digital transformation in higher education:a systematic review [J]. Big Data Research, 2023, 9(3): 56-70.
[2]	Hong MEI, Xiaoyong DU, Hai JIN, Xueqi CHENG, Yunpeng CHAI, Xuanhua SHI, Xiaolong JIN, Yasha WANG, Chi LIU. Big data technologies forward-looking [J]. Big Data Research, 2023, 9(1): 1-20.
[3]	Yang SHEN, Menglong YU. Metaverse and big data: data insight and value connection in spatio-temporal intelligence [J]. Big Data Research, 2023, 9(1): 103-110.
[4]	Jing CHEN. Humanities big data and its application in the field of digital humanities [J]. Big Data Research, 2022, 8(6): 3-14.
[5]	Yuchu LUO, Hao WU, Yuhan GUO, Shaocong TAN, Can LIU, Ruike JIANG, Xiaoru YUAN. Visualization in digital humanities [J]. Big Data Research, 2022, 8(6): 74-93.
[6]	Tongzheheng ZHENG, Bin LI, Minxuan FENG, Bolin CHANG, Dongbo WANG. Explore the structuration of historical books:the construction and quantitative analysis of digital humanities database of the Biographies of the Shiji [J]. Big Data Research, 2022, 8(6): 40-55.
[7]	Wenlong LI, Yuan YUAN, Xiaopeng AN. Modus operandi of big data governance: some preliminary observations [J]. Big Data Research, 2022, 8(4): 34-45.
[8]	Qifeng TANG, Zhiqing SHAO, Yazhen YE. Authenticating and licensing architecture of data rights in data trade [J]. Big Data Research, 2022, 8(3): 40-53.
[9]	Chenhuizi WANG, Wei CAI. Digital economics in metaverse: state-of-the-art, characteristics, and vision [J]. Big Data Research, 2022, 8(3): 140-150.
[10]	Mei YANG, Wei LI, Siyuan QIAO, Wei LIU. Research on calculation method of China’s big data industry output value [J]. Big Data Research, 2022, 8(3): 151-160.
[11]	Deren LI, Guo ZHANG, Yonghua JIANG, Xin SHEN, Weiling LIU. Opportunities and challenges of geo-spatial information science from the perspective of big data [J]. Big Data Research, 2022, 8(2): 3-14.
[12]	Xiaolan QIU, Yuxin HU, Songtao SHANGGUAN, Kun FU. Remote sensing satellite big data high-recision integration processing technology [J]. Big Data Research, 2022, 8(2): 15-27.
[13]	Weiquan LIU, Cheng WANG, Yu ZANG, Qian HU, Shangshu YU, Baiqi LAI. A survey on information extraction technology based on remote sensing big data [J]. Big Data Research, 2022, 8(2): 28-57.
[14]	Jianqiang LIU, Xiaomin YE, Youguo LAN. Remote sensing big data from Chinese ocean satellites and its application service [J]. Big Data Research, 2022, 8(2): 75-88.
[15]	Hequn YANG, Xiaofeng WANG, Yanqing GAO, Yiwen LU, Bingxin MA, Xinyao WANG. Analysis of satellite big data requirements in numerical weather prediction [J]. Big Data Research, 2022, 8(2): 89-102.

Design of big data processing system supporting multi-satellites and multi-tasks

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 13

Related Articles 15

Metrics

Recommended 0