基于关联规则的气象服务智能推荐

doi:10.11959/j.issn.2096-0271.2018020

Abstract

Abstract:

To overcome the problem in which public meteorological services are rarely optimized for personalization,a recommendation method incorporating improved association rules and collaborative filtering was presented.First,FPGrowth algorithm was applied on pre-processed Web log data to generate association rules.Meanwhile,a customized collaborative filtering algorithm was used to calculate pairwise similarities between meteorological products based on users' browsing records.A total of five experiments were conducted.The experimental results show that the rules are relatively accurate for precipitation and haze weather,because pairwise similarities between meteorological products which are calculated by the proposed algorithm could reduce the total number of invalid or irrelevant association rules by 10%.

Key words: intelligent recommendation, association rule, Web log, public meteorological service, parallel computing, similarity, collaborative filtering

CLC Number:

TP183

Wenfang ZHAO, Yanan LIU, Dongchang YU. Intelligent recommendation of meteorological service based on association rules[J]. Big Data Research, 2018, 4(2): 72-85.

Figures/Tables 14

References 22

[1]	纪淑娟, 王理, 梁永全 ,等. 基于神经网络的用户视频评分自动获取方法[J]. 计算机科学, 2015,42(11): 96-100.
	JI S J , WANG L , LIANG Y Q ,et al. Neuralnetwork-based method for automatic acquisition of user’s video rating[J]. Computer Science, 2015,42(11): 96-100.
[2]	王涛, 覃锡忠, 贾振红 ,等. 基于相似度和信任度的关联规则微博好友推荐[J]. 计算机应用, 2016,36(8): 2262-2267.
	WANG T , QIN X Z , JIA Z H ,et al. Association rules recommendation of microblog friend based on similarity and trust[J]. Journal of Computer Applications, 2016,36(8): 2262-2267.
[3]	LIU J G , ZHOU T , WANG B H . Personalized recommender systems:a survey of the state-of-the-art[J]. Progress in Natural Science, 2009,19(1): 1-15.
[4]	LIN W , ALVAREZ S A , RUIZ C . Efficient adaptive-support association rule mining for recommendation systems[J]. Data Mining and Knowledge Discovery, 2002,6(1): 83-105.
[5]	李杰, 徐勇, 王云峰 ,等. 面向个性化推荐的强关联规则挖掘[J]. 系统工程理论与实践, 2009,29(8): 144-152.
	LI J , XU Y , WANG Y F ,et al. Strongest association rules mining for personalized recommendation[J]. System EngineeringTheory ＆Practice, 2009,29(8): 144-152.
[6]	易芝, 汪琳琳, 王练 . 基于关联规则相关性分析的Web 个性化推荐研究[J]. 重庆邮电大学学报(自然科学版), 2007,19(2): 234-237.
	YI Z , WANG L L , WANG L . Research on Web personalized recommendation based on correlation analysis of association rule[J]. Journal of Chongqing University of Posts and Communications (Natural Science), 2007,19(2): 234-237.
[7]	鲍玉斌, 王大玲, 于戈 . 关联规则和聚类分析在个性化推荐中的应用[J]. 东北大学学报(自然科学版), 2003,24(12): 1149-1188.
	BAO Y B , WANG D L , YU G . Application of association rules and clustering analysis to personalized recommendation[J]. Journal of Northeastern University (Natural Science), 2003,24(12): 1149-1188.
[8]	张忠林, 田苗凤, 刘宗成 . 大数据环境下关联规则并行分层挖掘算法研究[J]. 计算机科学, 2016,43(1): 286-289.
	ZHANG Z L , TIAN M F , LIU Z C . Parallel hierarchical association rule mining in big data environment[J]. Computer Science, 2016,43(1): 286-289.
[9]	周诗慧, 殷建 . Hadoop 平台下的并行Web 日志挖掘算法[J]. 计算机工程, 2013,39(6): 43-46.
	ZHOU S H , YIN J . Parallel web log mining algorithm in Hadoop platform[J]. Computer Engineering, 2013,39(6): 43-46.
[10]	刘帅, 杨英杰, 常德显 ,等. 改进的模糊关联规则及其挖掘算法[J]. 计算机工程与设计, 2015,36(4): 942-947.
	LIU S , YANG Y J , CHANG D X ,et al. Improved fuzzy association rule and its mining algorithm[J]. Computer Engineer and Design, 2015,36(4): 942-947.
[11]	程广, 王晓峰 . 基于MapReduce的并行关联规则增量更新算法[J]. 计算机工程, 2016,42(2): 21-26.
	CHENG G , WANG X F . Incremental updating algorithm of parallel association rule based on MapReduce[J]. Computer Engineer, 2016,42(2): 21-26.
[12]	黄晓雯, 严明, 桑基韬 ,等. 基于关联规则挖掘的跨网络知识关联及协同应用[J]. 计算机科学, 2016,43(7): 51-57.
	HUANG X W , YAN M , SANG J T ,et al. Association rules mining based crossnetwork knowledge association and collaborative application[J]. Computer Science, 2016,43(7): 51-57.
[13]	陈平华, 陈传瑜, 洪英 . 一种结合关联规则的协同过滤推荐算法[J]. 小型微型计算机系统, 2016,37(2): 287-293.
	CHEN P H , CHEN C Y , HONG Y . Incorporating association rules for collaborative filtering recommendation algorithm[J]. Journal of Chinese Computer Systems, 2016,37(2): 287-293.
[14]	胡波, 黄宁, 仵伟强 . 基于业务路径和频度矩阵的关联规则挖掘算法[J]. 计算机科学, 2016,43(12): 146-154.
	HU B , HUANG N , WU W Q . Algorithm for mining association rules based on application paths and frequency matrix[J]. Computer Science, 2016,43(12): 146-154.
[15]	石岩, 邓敏, 刘启亮 ,等. 海陆气候事件关联规则挖掘方法[J]. 地理信息科学, 2014,16(2): 182-191.
	SHI Y , DENG M , LIU Q L ,et al. Discovering sequential association rules between single ocean climate index and land abnormal climate events[J]. Journal of Geo-information Science, 2014,16(2): 182-191.
[16]	刘娟, 宋安军 . 改进FP-growth 算法在气象预报中的应用[J]. 计算机系统应用, 2016,25(10): 199-205.
	LIU J , SONG A J . Application of an improve FP-growth algorithm in meteorological forecast[J]. Application of Computer System, 2016,25(10): 199-205.
[17]	BINZENH?FER A , TUTSCHKU K , GRABEN B A D ,et al. A P2P-based framework for distributed network management[M]. Berlin: Springer Berlin Heidelberg, 2005.
[18]	CHEVUL S , BINZENHFER A , SCHMID M ,et al. A self-organizing concept for distributed end-to-end quality monitoring[R]. Germany:University of Wurzburg Institute, 2006.
[19]	董长青, 任女尔, 张庆余 ,等. 基于HBase+ElasticSearch的海量交通数据实时存取方案设计[J]. 大数据, 2017(1): 80-89.
	DONG C Q , REN N E , ZHANG Q Y ,et al. Design scheme of massive traffic data real-time access based on HBase and Elastic Search[J]. Big Data Research, 2017(1): 80-89.
[20]	李大中, 刘剑, 邓景文 . 大数据能力开放平台创新和发展[J]. 大数据, 2017(1): 72-80.
	LI D Z , LIU J , DENG J W . Innovation and development of big data ability open platform[J]. Big Data Research, 2017(1): 72-80.
[21]	石勇, 孟凡 . 信用评分基本理论及其应用[J]. 大数据, 2017(1): 19-27.
	SHI Y , MENG F . Credit scoring:basic theory and applications[J]. Big Data Research, 2017(1): 19-27.
[22]	刘岩, 王华, 秦叶阳 ,等. 智慧城市多源异构大数据处理框架[J]. 大数据, 2017(1): 51-60.
	LIU Y , WANG H , QIN Y Y ,et al. Multisource heterogeneous big data processing architecture in smart city[J]. Big Data Research, 2017(1): 51-60.

Metrics

Recommended 0

No Suggested Reading articles found!

字段ID	字段含义
idvisitor	用户ID
id_url	页面ID
server_time	用户访问时间
time_spent_ref_action	页面停留时间
action_type	用户行为

字段ID	字段含义
id_url	页面ID
id_name	页面URL
id_re_name	去掉时间参数后的URL

字段ID	字段含义
id_url	页面ID
id_name	页面URL

字段ID	字段含义
id_url	页面ID
sim_url	相似度页面ID列表，ID之间用“$”分隔
sim_url_value	相似度列表，值之间用“$”分隔

操作系统	Red Hat 7.1
CPU	Intel(R) CPU E5-2680 v3，主频2.60 GHz，8核
内存	32 GB
硬盘	SAS 300 GB×2做的RAIDl，用于操作系统和软件；STAT l TB×3用于Hadoop 数据存储

Intelligent recommendation of meteorological service based on association rules

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 22

Related Articles 6

Metrics

Recommended 0

序号	配置参数
序号	参数1	参数2	参数3	spark.default.parallelism	耗时/s
1	2	1	1	未启用	138
2	4	8	2	未启用	126
3	8	8	2	未启用	44
4	12	6	8	未启用	38
5	16	4	8	未启用	33
6	20	4	8	280	27
7	24	3	2	900	19
8	24	2	3	1 000	23
9	24	3	3	1 200	26

实验组编号	规则	支持度	置信度
1	10,17]=>[3]	69.78%	98.52%
	10]=>[3]	62.14%	97.71%
	11]=>[824]	32.03%	97.34%
	13]=>[4]	38.65%	96.21%
	996]=>[824]	28.07%	95.6%
2	926]=>[3]	65.42%	96.32%
	16]=>[38]	62.06%	95.41%
	963，966]=> 4	38.79%	93.01%
	966，17]=> 824	26.41%	88.14%
	966，3]=> 824	23.01%	83.42%
3	18，3]=>[17]	46.13%	98.02%
	1020]=>[3]	37.98%	95.4%
	10]=>[824]	26.3%	92.1%
	15，11]=>[17]	24.32%	90.12%
	57，11]=>[17]	21.7%	89.03%
4	28，13]=>[17]	36.19%	93.58%
	1 019]=>[3]	33.98%	91.02%
	1 019，3]=>[824]	27.26%	88.21%
	1 018，11]=>[17]	24.03%	82.9%
	672，11]=>[17]	21.07%	79.16%
5	27，16]=>[982]	36.19%	93.58%
	672，45]=>[982]	33.98%	91.02%
	1 001，3]=>[675]	27.26%	88.21%
	673，3]=>[671]	24.03%	82.9%
	672,11]=>[17]	21.07%	79.16%

[1]	Zhitao ZHU, Shijing SI, Jianzong WANG, Jing XIAO. Survey on federated recommendation systems [J]. Big Data Research, 2022, 8(4): 105-132.
[2]	Yuzhe LIN, Weihua ZHANG. A research on GPU transactional memory [J]. Big Data Research, 2020, 6(4): 3-17.
[3]	Jianguo DENG, Sulan ZHANG, Jifu ZHANG, Yaling XUN, Aiqin LIU. Loss function and application research in supervised learning [J]. Big Data Research, 2020, 6(1): 60-80.
[4]	Cuiping Li, Mengwei Lan, Benyou Zou, Shaoqing Wang, Kankan Zhao. Big Data and Recommendation System [J]. Big Data Research, 2015, 1(3): 23-35.
[5]	Yun Xiong, Yangyong Zhu. Abnormal Group Mining:Framework and Applications [J]. Big Data Research, 2015, 1(2): 66-77.
[6]	Yihua Huang. Research Progress on Big Data Machine Learning System [J]. Big Data Research, 2015, 1(1): 35-54.