Big Data Research ›› 2020, Vol. 6 ›› Issue (4): 18-29.doi: 10.11959/j.issn.2096-0271.2020030

• TOPIC:HETEROGENEOUS PARALLEL SYSTEMS FOR BIG DATA • Previous Articles     Next Articles

Design,implementation and practice of parallel processing system for a large-scale heterogeneous data

Zhengxun XIA,Shengmei LUO,Yuanhao SUN,Jianfei TANG,Yan ZHANG   

  1. Transwarp Information Technology (Shanghai) Co.,Ltd.,Shanghai 200233,China
  • Online:2020-07-15 Published:2020-07-18

Abstract:

With the rapid development of Internet and IoT applications,data processing has gradually expanded from structured to structured,semi-structured and unstructured hybrid heterogeneous data processing mode.A large-scale heterogeneous data parallel processing system was designed.Based on the functional view of a unified platform,the unified resource management framework was adopted to store and query a variety of heterogeneous data,including structured,JSON/XML,graph data,document data,etc.Adopting a unified database language,the parallel computing across data types and database engines was realized,and the needs of multi business application development were met.The feasibility of the system is verified by standard evaluation environment and commercial deployment.

Key words: parallel system, heterogeneous data, unified architecture

CLC Number: 

No Suggested Reading articles found!