大数据 ›› 2016, Vol. 2 ›› Issue (3): 73-82.doi: 10.11959/j.issn.2096-0271.2016032

• 专题:大数据与智慧城市 • 上一篇    下一篇

基于HBase的海量GIS数据分布式处理实践

李雪梅1,邢俊峰1,刘大伟1,王海洋1,2,刘玮1,2   

  1. 1 烟台中科网络技术研究所,山东 烟台 264003
    2 中国科学院计算技术研究所,北京 100080
  • 出版日期:2016-05-20 发布日期:2017-03-08
  • 基金资助:
    山东省自主创新及成果转化专项基金资助项目;山东省科技发展计划基金资助项目;烟台市科技发展计划基金资助项目

Distributed processing practice of the massive GIS data based on HBase

Xuemei LI1,Junfeng XING1,IUDawei L1,Haiyang WANG1,2,Wei LIU1,2   

  1. 1 Institute of Network Technology,ICT(YANTAI),Yantai 264003,China
    2 Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100080,China
  • Online:2016-05-20 Published:2017-03-08
  • Supported by:
    Technology Innovation Program of Shandong Province of China;Shandong Technology Research and Development Program;Yantai Technology Research and Development Program

摘要:

设计了一种基于分布式数据库HBase的GIS数据管理系统。系统优化了栅格数据的生成和存储过程,将海量栅格数据直接写入HBase存储、索引。同时,针对矢量空间数据的存储、索引与检索,提出了一种新的rowkey设计,既考虑经纬度,又考虑空间数据类型和属性,使得在按空间位置检索矢量地理信息时,能通过HBase的rowkey迅速定位需要返回的数据。在HBase的集群环境上用真实GIS数据对上述方法进行了验证,结果表明,提出的系统具有较高的海量数据存储和检索性能,实现了海量地理信息数据的高效存储和实时高速检索。

关键词: 大数据, HBase, 栅格数据, 矢量数据, rowkey

Abstract:

Based on the distributed database HBase,a kind of GIS data management system was designed.The system optimized the generated and stored procedures of raster data,which could be directly written into the storage and indexing of the HBase.At the same time,in view of the storing,indexing and retrieval of the vector spatial data,a new design for rowkey was proposed that considering both the latitude and longitude,and the spatial data types and attributes.So that the data needed to be returned could be quickly located by rowkey of the HBase,when retrieving vector geographic information according to the spatial location.The above methods had been verified on the HBase cluster environment with real GIS data.The results show that the proposed system has high performance for storage and retrieval of mass data,and realizes the efficient storage and real-time high-speed retrieval of the vast geographic information data.

Key words: big data, HBase, raster data, vector data, rowkey

No Suggested Reading articles found!