数据管道模型：场外流式数据市场形态探索

doi:10.11959/j.issn.2096-0271.2023031

Abstract

Abstract:

The current exploration of data element market construction mainly focuses on the construction of data trading venues, while the streaming data market refers to the market in which data suppliers continuously and quickly supply specific data to data users.Streaming data is not suitable for transactions in the data trading venues, so the over-the-counter trading model of streaming data needs to be explored.After studying the current operation status of the streaming data market, it was pointed out that market disorder and insufficient regulatory tools were the main problems, and a data pipeline model for the over-the-counter streaming data market was proposed, including pipeline circulation elements (data pipeline, data factory, data supply chain), market regulation elements (data meter, quality sampling device, compliance auditing device), etc.The technical feasibility of the data pipeline model was demonstrated, and it provided theoretical and technical support for the construction, regulation and supervision of the over-the-counter data market.

Key words: data market, data pipeline, streaming data, over-the-counter transaction

CLC Number:

TP311

Hongrun REN, Yangyong ZHU. Data pipeline model: exploration of the overthe-counter form of streaming data[J]. Big Data Research, 2023, 9(3): 15-28.

Figures/Tables 4

References 23

[1]	包晓丽, 杜万里 . 数据可信交易体系的制度构建——基于场内交易视角[J]. 电子政务, 2023:2023.06.003.
	BAO X L , DU W L . Institutional construction of data credible trading system—based on the perspective of floor trading[J]. E-Government, 2023:2023.06.003.
[2]	CHEN J , LI M , XU H . Selling data to a machine learner:pricing via costly signaling[C]// Proceedings of the 39th International Conference on Machine Learning.[S.l.:s.n.], 2022: 3336-3359.
[3]	CONG Z C , LUO X , PEI J ,et al. Data pricing in machine learning pipelines[J]. Knowledge and Information Systems, 2022,64(6): 1417-1455.
[4]	HERNANDEZ D , GAMEIRO L , SENNA C ,et al. Handling producer and consumer mobility in IoT publish–subscribe named data networks[J]. IEEE Internet of Things Journal, 2022,9(2): 868-884.
[5]	KOLAJO T , DARAMOLA O , ADEBIYI A . Big data stream analysis:a systematic literature review[J]. Journal of Big Data, 2019,6(1): 1-30.
[6]	KARMAKAR G , CHOWDHURY A , DAS R ,et al. Assessing trust level of a driverless car using deep learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2021,22(7): 4457-4466.
[7]	ZHANG P , LIU B X , LU T ,et al. A semantic embedding enhanced topic model for user-generated textual content modeling in social ecosystems[J]. The Computer Journal, 2022,65(11): 2953-2968.
[8]	O’CALLAGHAN L , MISHRA N , MEYERSON A ,et al. Streamingdata algorithms for high-quality clustering[C]// Proceedings of 18th International Conference on Data Engineering. Piscataway:IEEE Press, 2002: 685-694.
[9]	孙大为, 张广艳, 郑纬民 . 大数据流式计算:关键技术及系统实例[J]. 软件学报, 2014,25(4): 839-862.
	SUN D W , ZHANG G Y , ZHENG W . Big data stream computing:technologies and instances[J]. Journal of Software, 2014,25(4): 839-862.
[10]	LI Y , YANG J C , ZHANG Z ,et al. Healthcare data quality assessment for cybersecurity intelligence[J]. IEEE Transactions on Industrial Informatics, 2023,19(1): 841-848.
[11]	WU X B , XU Y H , SHAO Z L ,et al. LSM-trie:an LSM-tree-based ultralarge key-value store for small data[C]// Proceedings of the 2015 USENIX Conference on USENIX Annual Technical Conference. New York:ACM Press, 2015: 71-82.
[12]	蔡莉, 梁宇, 朱扬勇 ,等. 数据质量的历史沿革和发展趋势[J]. 计算机科学, 2018,45(4): 1-10.
	CAI L , LIANG Y , ZHU Y Y ,et al. History and development tendency of data quality[J]. Computer Science, 2018,45(4): 1-10.
[13]	NIU C Y , ZHENG Z Z , WU F ,et al. Online pricing with reserve price constraint for personal data markets[C]// Proceedings of 2020 IEEE 36th International Conference on Data Engineering. Piscataway:IEEE Press, 2020: 1978-1981.
[14]	陈纯 . 流式大数据实时处理技术、平台及应用[J]. 大数据, 2017,3(4): 1-8.
	CHEN C . Real-time processing technology,platform and application of streaming big data[J]. Big Data Research, 2017,3(4): 1-8.
[15]	KREPS J , NARKHEDE N , RAO J . Kafka:a distributed messaging system for log processing[C]// Proceedings of the NetDB.[S.l.:s.n.], 2011.
[16]	MISRA S , REISSLEIN M , XUE G L . A survey of multimedia streaming in wireless sensor networks[J]. IEEE Communications Surveys ＆ Tutorials, 2008,10(4): 18-39.
[17]	TOSHNIWAL A , TANEJA S , SHUKLA A ,et al. Storm@twitter[C]// Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2014: 147-156.
[18]	CARBONE P , KATSIFODIMOS A , EWEN S ,et al. Apache Flink:stream and batch processing in a single engine[J]. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015,36(4): 28-38.
[19]	ALI M H , CHANDRAMOULI B , FAY J ,et al. Online visualization of geospatial stream data using the worldwide telescope[J]. Proceedings of the VLDB Endowment, 2011,4(12): 1379-1382.
[20]	WANG G Z , CHEN L , DIKSHIT A ,et al. Consistency and completeness:rethinking distributed stream processing in Apache Kafka[C]// Proceedings of the 2021 International Conference on Management of Data. New York:ACM Press, 2021: 2602-2613.
[21]	LIU Q , FENG G Z , ZHENG W B ,et al. Managing data quality of cooperative information systems:model and algorithm[J]. Expert Systems With Applications, 2022,189.
[22]	AN J , WU S Y , GUI X L ,et al. A blockchainbased framework for data quality in edgecomputing-enabled crowdsensing[J]. Frontiers of Computer Science, 2023,17(4): 174503.
[23]	BODON F , RóNYAI L . Trie:an alternative data structure for data mining algorithms[J]. Mathematical and Computer Modelling, 2003,38(7-9): 739-751.

Metrics

Recommended 0

No Suggested Reading articles found!

Data pipeline model: exploration of the overthe-counter form of streaming data

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 23

Related Articles 5

Metrics

Recommended 0

[1]	Yazhen YE, Yangyong ZHU. Data-Commerce-Ecosystem: data goods, data businessman and data commerce [J]. Big Data Research, 2023, 9(1): 111-125.
[2]	Keman HUANG, Xiaoyong DU. Value chain model of data governance and its application on data governance regulation analysis [J]. Big Data Research, 2022, 8(4): 3-16.
[3]	Yazhen YE, Yangyong ZHU. BoxedData: a data product form based on databox [J]. Big Data Research, 2022, 8(3): 15-25.
[4]	Xiangqian DONG, Bing GUO, Yan SHEN, Xuliang DUAN, Yuncheng SHEN, Hong ZHANG. Data assets value evaluation model based on profit maximization [J]. Big Data Research, 2020, 6(3): 13-20.
[5]	Jingwei WANG, Zhenzhe ZHENG, Fan WU, Guihai CHEN. Blockchain based data marketplace [J]. Big Data Research, 2020, 6(3): 21-35.