基于数据仓库的高校数据统计服务平台研究

doi:10.3969/j.issn.1000-436x.2013.Z2.031

摘要/Abstract

摘要：

为了适应新形势下高校对大数据统计、分析的需求，设计实现了一套基于数据仓库的高校数据统计服务平台的整体方案。对高校数据仓库的整体架构、数据仓库的建模进行了研究，井设计了一种基于数据仓库星型结构的统计数据模型；进一步探讨了统计服务平台在实现过程中ETL(extraction transformation loading)以及界面设计等方面的关键技术；为高校下一步搭建管理与决策支持平台提供了重要的实践基础和理论支撑。

关键词: 商业智能, 数据统计, 数据仓库, 星型模型, 统计数据模型

Abstract:

In order to adapt to the demand of statisting and analyzing huge-data in universities under current situation,an overall programme was designed and implemented to construct data statistical service platform based on data warehouse.The overall framework of university data warehouse and the modeling method of data warehouse were researched,and a statistical data model based on data star-schema model was designed.Furthermore,the key technology in the process of implementing statistical service platform like ETL(Extraction-Transformation-Loading) and UI interface design were explored.It provides practical basis and theoretical support for the next step to build the management and decision support platform of universities.

Key words: business intelligence, data statistics, data warehouse, star-schema model, statistics data model

龙新征,李丽,彭一明,王倩宜,欧阳荣彬,刘云峰. 基于数据仓库的高校数据统计服务平台研究[J]. 通信学报, 2013, 34(Z2): 163-169.

Xin-zheng LONG,Li LI,Yi-ming PENG,Qian-yi WANG,Rong-bin OUYANG,Yun-feng LIU. Research of university data statistical platform based on data warehouse[J]. Journal on Communications, 2013, 34(Z2): 163-169.

图/表 7

图1

图2

图3

图4

表1

图5

图6

参考文献 9

[1]	DRESNER H . Profiles in Performance:Business Intelligence Journeys and the Roadmap for Change[M]. New York: John Wiley and SonsPress, 2009.
[2]	罗永辉, 陈明亮 . 商业智能皙学思索-BI框架体系的另一种视角[J]. 软件世界, 2007,7: 68-71. LUO Y H , CHEN M L . The philosophic thinking of business intelligence-another view of the architecture of BI[J]. Software World, 2007,7: 68-71.
[3]	WASSERMAN T J , MARTIN P , SKILLICORN D B ,et al. Developing a characterization of business intelligence workloads for sizingnew database systems[A]. Proceedings of the 7th ACM International Workshop on Data Warehousing and OLAP[C]. 2004. 7-13.
[4]	INMON W H . Building the Data Warehouse[M]. New York: John Wiley and SonsPress, 2003.
[5]	王倩宜, 李丽, 欧阳荣彬 ,等. 数据综合服务管理平台主体方案探讨[J]. 实验技术与管理, 2011,28(5): 4-6. WANG Q Y , LI L , OUYANG R B ,et al. Discussion on infrastructure proposal of data integration and service platform[J]. Experimental Technology and Management, 2011,28(5): 4-6.
[6]	马宁, 曹震, 李军 . 基于数据仓库的银行监管报表系统研究[J]. 软件导刊, 2013,12(4): 137-141. MA N , CAO Z , LI J . The research on the bank superintendency reporting system based on data warehouse[J]. Software Guide, 2013,12(4): 137-141.
[7]	KIMBALL R , ROSS M . The Data Warehouse Toolkit:The Complete Guide to Dimensional Modeling[M]. New York: John Wiley and SonsPress, 2002.
[8]	高亮 . 高校数据仓库建设研究[J]. 武汉大学学报(理学版), 2012,58(S1): 179-184. GAO L . The research on the construction of data warehouse in universities[J]. Journal of Wuhan University(Natural Science Edition), 2012,58(S1): 179-184
[9]	王亦然 . 数据仓库技术基于业务报表数据的应用研究[D]. 北京:中国地质大学(北京), 2010. WANG Y R . The Research of Application Business Reporting Data Based on Data Warehouse Technology[D]. Beijing:China University of Geosciences (Beijing), 2010.

主要问题	表现形式	产生原因	清洗策略
主要问题	大量空值字段的出现	源系统中对很多字段没有做非空限制	1) 交由源系统重新录入、补齐2) 在数据仓库对应的维表中建立一个新的字段，将这些空值字段的值统一赋值
完整性问题	超出字典表范围	填写这些值的时候是直接让用户填写而非下拉框选择	1) 交由源系统重新录入、补齐2) 在数据仓库对应的维表中建立一个新的字段，将这些空值字段的值统一赋值
一致性问题	一个特定的字段在不同的表中内容不同应该成为主键的值不唯一	录入，同步的问题源系统中未建立有效的主键关系	1) 交由源系统重新录入、修改2) 选取最可靠的表中的字段为确定值，消除错误、重复的主键