从数据质量到数据产品质量

doi:10.11959/j.issn.2096-0271.2022040

Abstract

Abstract:

For a long time, the purpose of data quality research is to fulfill requirements of the normal operation of the organization’s own information system.With the construction and development of data market, the requirements on data quality have changed from “self-use” to “other use” and “need for supervision”.The data products quality in the data market is the focus of data users (buyers) and market regulators.The demands of users and regulators for data product quality were analyzed, and a framework of data product quality was proposed innovatively.On this basis, taking BoxedData as an example, the corresponding quality dimensions, quality indicators and quality assessment models were construct from three aspects of time, space and content integrity.The quality framework was suitable for detecting and assessing resource data products, and could provide effective detection methods and standards for data product buyers and market regulators.

Key words: data product, data quality, quality control, BoxedData

CLC Number:

TP301

Li CAI, Yangyong ZHU. From data quality to data products quality[J]. Big Data Research, 2022, 8(3): 26-39.

Figures/Tables 4

References 26

[1]	蔡莉, 朱扬勇 . 大数据质量[M]. 上海: 上海科学技术出版社, 2017.
	CAI L , ZHU Y Y . The quality of big data[M]. Shanghai: Shanghai Scientific ＆Technical Publishers, 2017.
[2]	SCANNAPIECO M , CATARCI T . Data quality under the computer science perspective[J]. Archivi ＆ Computer, 2002,2: 1-13.
[3]	蔡莉, 梁宇, 朱扬勇 ,等. 数据质量的历史沿革和发展趋势[J]. 计算机科学, 2018,45(4): 1-10.
	CAI L , LIANG Y , ZHU Y Y ,et al. History and development tendency of data quality[J]. Computer Science, 2018,45(4): 1-10.
[4]	WANG R Y . A product perspective on total data quality management[J]. Communications of the ACM, 1998,41(2): 58-65.
[5]	涂子沛 . 大数据[M]. 桂林: 广西师范大学出版社, 2013.
	TU Z P . The big data revolution[M]. Guilin: Guangxi Normal University Press, 2013.
[6]	王军玲, 李华, 王强 . ISO 8000数据质量系列标准探析[J]. 标准科学, 2010(12): 44-46.
	WANG J L , LI H , WANG Q . Research on ISO 8000 series standards for data quality[J]. Standard Science, 2010(12): 44-46.
[7]	宋立荣, 彭洁 . 美国政府“信息质量法”的介绍及其启示[J]. 情报杂志, 2012,31(2): 12-18.
	SONG L R , PENG J . Introduction and inspirations of the “information quality act” in the American federal government[J]. Journal of Intelligence, 2012,31(2): 12-18.
[8]	朱扬勇, 熊贇 . 数据的经济活动及其所需要的权利[J]. 大数据, 2020,6(6): 140-150.
	ZHU Y Y , XIONG Y . The required authorization to the data-centric economic activities[J]. Big Data Research, 2020,6(6): 140-150.
[9]	DENBALY M , GLASER L , VASAVADA U . ERS data product quality standards[Z]. 2021.
[10]	孙苗, 王子珂, 童心 ,等. 典型海洋环境观测数据产品应用现状及对我国的启示[J]. 大数据, 2022,8(1): 73-83.
	SUN M , WANG Z K , TONG X ,et al. Status of classical marine environmental observing data products application and enlightenment to China[J]. Big Data Research, 2022,8(1): 73-83.
[11]	吴琼, 白廷俊, 庞清珊 . 上海数据交易所今日揭牌成立[N]. 北京日报, 2021-11-25.
	WU Q , BAI T J , PANG Q S . Shanghai Data Exchange was inaugurated today[N]. Beijing Daily, 2021-11-25.
[12]	CAI L , WANG H Y , SHA C ,et al. The mining of urban hotspots based on multisource location data fusion[J]. IEEE Transactions on Knowledge and Data Engineering, 2021(99): 1.
[13]	闫鑫, 黄国彬 . 科学数据分类研究述评[J]. 图书馆论坛, 2020,40(5): 45-54.
	YAN X , HUANG G B . A review of the study of research data classification[J]. Library Tribune, 2020,40(5): 45-54.
[14]	CAI L , ZHU Y Y . The challenges of data quality and data quality assessment in the big data era[J]. Data Science Journal, 2015,14:2.
[15]	VANDENBERGHE W , VANHAUWAERT E , DEMEESTER P ,et al. Feasibility of expanding traffic monitoring systems with floating car data technology[J]. IET Intelligent Transport Systems, 2012,6(4): 347-354.
[16]	程静 . 微天气与时间、位置和品类的交互对移动O2O平台商家销量的影响研究[D]. 哈尔滨:哈尔滨工业大学, 2021.
	CHENG J . Research on the impact of the interaction of micro-weather with time,location and category on the sales of merchants on mobile O2O platform[D]. Harbin:Harbin Institute of Technology, 2021.
[17]	黄明峰, 刘军, 靖剑波 . 贵阳市政府数据开放平台设计与实现[J]. 电信科学, 2017,33(9): 136-147.
	HUANG M F , LIU J , JING J B . Design and practice of Guiyang open government data platform[J]. Telecommunications Science, 2017,33(9): 136-147.
[18]	相丽玲, 贾昆 . 中外个人数据保护标准研究进展与未来趋势分析[J]. 情报杂志, 2020,39(2): 85-94.
	XIANG L L , JIA K . Research progress and future trend of personal data protection standards at home and abroad[J]. Journal of Intelligence, 2020,39(2): 85-94.
[19]	李延舜 . 隐私政策在企业数据合规实践中的功能定位[J]. 江汉论坛, 2020(10): 136-144.
	LI Y S . Privacy policy’s functional orientation in the compliance practice of corporate data[J]. Jianghan Tribune, 2020(10): 136-144.
[20]	刘长玉 . 政府、第三方检测机构和企业质量监管中博弈关系研究[J]. 东岳论丛, 2015,36(10): 128-132.
	LIU C Y . Study on the game relationship between government,third-party testing institutions and enterprise quality supervision[J]. Dongyue Tribune, 2015,36(10): 128-132.
[21]	陈红玉, 翟军, 袁长峰 ,等. 开放政府数据的溯源元数据研究及应用[J]. 情报杂志, 2017,36(6): 148-155.
	CHEN H Y , ZHAI J , YUAN C F ,et al. Provenance metadata research of open government data and application discussion[J]. Journal of Intelligence, 2017,36(6): 148-155.
[22]	胡韵, 胡爱群, 胡奥婷 ,等. 大数据背景下数据可追踪性应用分析与方法研究[J]. 密码学报, 2020,7(5): 565-582.
	HU Y , HU A Q , HU A T ,et al. Applications analysis and methods research of data traceability in big data content[J]. Journal of Cryptologic Research, 2020,7(5): 565-582.
[23]	叶雅珍, 朱扬勇 . 盒装数据:一种基于数据盒的数据产品形态[J]. 大数据, 2022,8(3): 15-25.
	YE Y Z , ZHU Y Y . BoxedData:a data product form based on databox[J]. Big Data Research, 2022,8(3): 15-25.
[24]	MCGILVRAY D . Executing data quality projects:ten steps to quality data and trusted information TM[M]. San Francisco: Morgan Kaufmann Publishers, 2008.
[25]	蔡莉, 李永轩, 王淑婷 ,等. 基于层次分析法的众源地理数据质量评估研究[J]. 测绘地理信息, 2021,46(3): 98-102.
	CAI L , LI Y X , WANG S T ,et al. Quality assessment for crowdsourcing geographic data using AHP[J]. Journal of Geomatics, 2021,46(3): 98-102.
[26]	任艳, 易宝林, 陈佳丽 . 基于V9I模型的空间拓扑一致性发现与维护[C]// 第二十三届中国数据库学术会议论文集. 出版地不详:出版机构不详, 2006: 400-402.
	REN Y , YI B L , CHEN J L . Spatial topological consistency discovering and maintenance based on V9I model[C]// Proceedings of the 23rd National Database Conference of China.[S.l.:s.n.], 2006: 400-402.

Metrics

Recommended 0

No Suggested Reading articles found!

质量维度	质量指标	定义
时间完整性	时间覆盖率	数据对象在数据产品所描述的各个时间点上的完整程度
	时效性	数据产品从产生到获取再到利用的一个很显著的时间差值
	可溯源性	是否提供数据产品在生命周期内所经历的操作及变换信息的描述
空间完整性	空间覆盖率	数据对象包含在数据产品所描述的空间中的完整程度
	空间一致性	不同的数据对象在空间上的逻辑关系是否正确和完整
内容完整性	属性覆盖率	数据产品在属性集上的完整程度
	准确性	数据产品的正确性、可靠性和可鉴别的程度
	一致性	数据对象在不同属性上的逻辑关系是否正确和完整，或者不同对象之间的逻辑关系是否正确和完整
	可获取性	数据产品可以被获取或者允许授权用户进行下载和使用的方便程度
	权威性	数据产品来源真实且可靠的程度

变量名	定义
P	数据产品
o	数据对象，P={o₁, o₂, o₃,…, o_N}
N	数据对象的数量
A	数据产品的属性集，A = {A₁, A₂, A₃,…, A_M}
M	属性集的数量
K	数据产品的时间范围，将其切分为K个时间点
LN	空间数据产品中数据对象所在层的数量

From data quality to data products quality

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 26

Related Articles 15

Metrics

Recommended 0

[1]	Yan ZHANG, Yifan YANG, Ren YI, Shengmei LUO, Jianfei TANG, Zhengxun XIA. Exploration and practice of data quality governance in privacy computing scenarios [J]. Big Data Research, 2022, 8(5): 55-73.
[2]	Lihua HUANG, Yifan DOU, Mengke GUO, Qifeng TANG, Gen LI. Features and transaction modes of data products in data markets [J]. Big Data Research, 2022, 8(3): 3-14.
[3]	Yazhen YE, Yangyong ZHU. BoxedData: a data product form based on databox [J]. Big Data Research, 2022, 8(3): 15-25.
[4]	Wen SU, Li ZHANG, Xuebing GUO, Honglin HE, Xinzhai TANG, Xiaoli REN. Classification of ecosystem long term observation data product [J]. Big Data Research, 2022, 8(1): 84-97.
[5]	Nan LIU, Xuejing HAO, Yuhong CHEN. A review and comparative analysis of domestic and foreign research on big data pricing methods [J]. Big Data Research, 2021, 7(6): 89-102.
[6]	Yu ZHANG, Haibo YUAN, Yanping WANG, Shaowu DONG, Jihai ZHANG. Management control and application of the time-frequency scientific data [J]. Big Data Research, 2021, 7(6): 120-127.
[7]	Hong DAI, Qun ZHANG, Haolin LU, Junzhi BIN. Comparative analysis between bank industry data governance guidelines and DCMM [J]. Big Data Research, 2020, 6(5): 118-128.
[8]	Yazhen YE, Guohua LIU, Yangyong ZHU. An initial exploration on framework of data assetization [J]. Big Data Research, 2020, 6(3): 3-12.
[9]	Xiaoou DING, Hongzhi WANG, Shengjian YU. Data quality management of industrial temporal big data [J]. Big Data Research, 2019, 5(6): 19-29.
[10]	Xiaoyuan BAO, Kai ZHANG, Meng JIN, Shuanglian XIE, Kai SONG. A data-space based platform for the integration and application of electronic health records [J]. Big Data Research, 2019, 5(6): 47-61.
[11]	Pingwen ZHANG, Weinan E, Xiaoru YUAN, Yiming FU. Big data analysis and application technology innovation platform [J]. Big Data Research, 2018, 4(4): 86-93.
[12]	Guangyong ZHENG, Zhen YANG, Ruifang CAO, Wan LIU, Yixue LI, Guoqing ZHANG. Quality control of big data analysis for metagenomics [J]. Big Data Research, 2018, 4(3): 3-12.
[13]	Qi YE, Liang ZHAO, Tong RUAN, DongLei FENG, Ju GAO, Min LIU. Usability research of regional health data for clinical efficacy analysis [J]. Big Data Research, 2018, 4(3): 13-23.
[14]	Tong RUAN, Ju GAO, Donglei FENG, Xiyuan QIAN, Ting WANG, Chenglin SUN. Process and methods of clinical big data mining based on electronic medical records [J]. Big Data Research, 2017, 3(5): 83-98.
[15]	Chunyu JIANG,Kai WEI. Basic capability and performance test of big data platform [J]. Big Data Research, 2017, 3(4): 37-45.