大数据中基于时态特征和混合式搜索的博客筛选挖掘

doi:10.11959/j.issn.1000-0801.2017001

Abstract

Abstract:

Concerning that the correlation degree of the existing methods of blog screen and mining is loose and the information retrieval of the methods is deficient,a method based on temporal feature and hybrid search method was proposed.Considering the user reviews are important sources of evidence combination,the average number of reviews for blogs,the sources of BM25 relevance scores,the longest blog BM25 scores and time range between the latest related blog paper and the oldest related blog paper are being as the temporal feature sets.In addition,considering local search advantage of linear search(LS) and global search advantage of differential evolution(DE),the two kinds of information search methods were combined.BlogS06 data set was used in the experiment which was consists of blog home pages,XML source files and its blog portal pages,it was used for TREC 2007 and TREC 2008 blog mining experiments.Experimental results show that the proposed method can obtain satisfactory results in terms of running time and effectiveness.

Key words: blog screening and mining, temporal feature, linear search, differential evolution, big data, BM25

CLC Number:

TP391

Lina ZHANG,Tai KUANG,Diqing JIANG. Blog screening and mining based on temporal features and hybrid search in big data[J]. Telecommunications Science, 2017, 33(1): 77-84.

Figures/Tables 6

References 17

[1]	RUCCELL M A . 社交网站的数据挖掘与分析[M]. 苏统华，魏通，赵逸雪，等译. 北京：机械工业出版社， 2015.
	RUCCELL M A . Mining the social web[M]. SU T H， WEI T， ZHAO Y X， et al. Beijing: China Machine Press, 2015.
[2]	关静怡 . 高质量博客检索中核心技术的研究[D]. 北京：北京邮电大学， 2011.
	GUAN J Y . Research on core technology of high quality blog retrieval[D]. Beijing: Beijing University of Posts and Telecommunications, 2011.
[3]	MACDONALD C , OUNIS I , SOBOROFF I . Overview of the TREC 2007 blog track [C]//16th Text Retrieval Conference, November6-9,2007, Gaithersburg,Maryland,USA. New Jersey: IEEE Press, 2007:1908-1910.
[4]	翟姗姗，许鑫，夏立新 . 学术博客中的用户交流与知识传播研究述评[J]. 现代图书情报技术， 2015,31(Z1): 3-12.
	ZHAI S S , XU X , XIA L X . Review of the research on user communication and knowledge dissemination in academic blogs[J]. New Technology of Library and Information Service, 2015,31(Z1): 3-12.
[5]	ELSAS J L , ARGUELLO J , CALLAN J , et al. Retrieval and feedback models for blog feed search [C]// International ACM SIGIR Conference on Research and Development in Information Retrieval, July20-24,2008, Singapore. New York: ACM Press, 2008:347-354.
[6]	林旺，翁彧 . 一种面向博客群的主题倾向性分析模型[J]. 中央民族大学学报(自然科学版)， 2014,23(3): 33-37.
	ZHAI S S , XU X , XIA L X . Review of the research on user communication and knowledge dissemination in academic blogs[J]. New Technology of Library and Information Service, 2015,31(Z1): 3-12.
[7]	郑美玉 . 基于本体的中文博客二级自动分类研究[J]. 情报科学， 2016,34(2): 87-90.
	ZHENG M Y . Research on two level automatic classification of Chinese blogs based on ontology[J]. Information Science, 2016,34(2): 87-90.
[8]	于航 . 基于图模型的博客排序系统的研究与实现[D]. 北京：北京大学， 2011.
	YU H . Research and implementation of blog ranking system based on graph model[D]. Beijing: Beijing University, 2011.
[9]	PARAPAR J , VIDAL M , SANTOS J . Finding the best parameter setting:particle swarm optimization [C]//The 2nd Spanish Conference on Information Retrieval (CERI 2012), June18-19,2012, Valencia,Spain. New Jersey: IEEE Press, 2012:49-60.
[10]	BOLLEGALA D , NOMAN N , IBA H . RankDE:learning a ranking function for information retrieval using differential evolution [C]// Conference on Genetic and Evolutionary Computation, July12-16,2011, Dublin,Ireland. New York: ACM Press, 2011:1771-1778.
[11]	LIN C , LIN C , LIN Z Y , et al. Hybrid pseudo-relevance feedback for microblog retrieval[J]. Journal of Information Science, 2013,39(6):773-788.
[12]	范晨熙，黄理灿，李雪利 . 基于 Lucene 的 BM25模型的评分机制的研究[J]. 工业控制计算机， 2013,26(3): 78-79.
	FAN C X , HUANG L C , LI X L . Research on scoring mechanism of BM25 model based on Lucene[J]. Industrial Control Computer, 2013,26(3): 78-79.
[13]	LAVRENKO V , CROFT W B . Relevance based language models [C]//International ACM SIGIR Conference on Research and Development in Information Retrieval, September9-13,2001, New Orleans,USA. New York: ACM Press, 2001:120-127.
[8]	付仅 . 论博客证据[D]. 重庆：重庆邮电大学， 2013.
	FU J . Study on blog evidence[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2013.
[15]	ZHANG S B , ZHANG B , ZHANG Y , et al. A search log sparseness oriented query expansion method [C]// International Conference on Systems and Informatics, November15-17,2014, Shanghai,China. New Jersey: IEEE Press, 2014:1050-1055.
[16]	许斌，亓晋，印溪，等. 基于多策略离散差分进化的移动互联网个性化服务组合[J]. 电信科学， 2016,32(2): 1045-1051.
	XU B , QI J , YIN X , et al. Personalized service composition based on multi-strategy discrete differential evolution in mobile internet[J]. Telecommunications Science, 2016,32(2): 1045-1051.
[17]	MACDONALD C , OUNIS I . The TREC blogs06 collection:creating and analysing a blog test collection[EB/OL]. 2016-02-29 2016-05-27.

Metrics

Recommended 0

No Suggested Reading articles found!

参数	数值
独立博客数目	100、649
RSS	62%
Atom	38%
首次消息来源	2013-12-06
末次消息来源	2014-02-21
提取消息来源数目	753、681
永久链接数目	3、215、171
主页数目	324、880
未压缩总大小/GB	148
消息来源（未压缩）/GB	38.6
永久链接（未压缩）/GB	88.8
主页（未压缩）/GB	2.8

查询集	训练次数/次	测试次数/次
2007a	974～995	951～973
2007b	951～973	974～995
2008a	1 076～1 100	1 051～1 075
2008b	1 051～1 075	1 076～1 100

方法	2007a		2007b
方法	MAP	P@10	MAP	P@10
第2.3节（2）所提方法	0.42	0.53	0.46	0.44
第2.4节（4）所提方法	0.40	0.51	0.47	0.43
线性搜索	0.52	0.60	0.56	0.52
差分进化	0.53	0.60	0.47	0.51
粒子群优化	0.52	0.58	0.55	0.59
提出的方法	0.59	0.67	0.59	0.69

方法	2008a		2008b
方法	MAP	P@10	MAP	P@10
第2.3节（2）所提方法	0.35	0.35	0.44	0.46
第2.4节（4）所提方法	0.32	0.29	0.36	0.38
线性搜索	0.54	0.35	0.55	0.46
差分进化	0.55	0.36	0.56	0.47
粒子群优化	0.52	0.35	0.54	0.46
提出的方法	0.57	0.37	0.58	0.56

[1]	Lu HAN, Weiyu CHEN, Fei ZHANG, Jianfeng HE, Huaizhen SU. Big data classification method of non relational distributed submission information under differentiated requirements [J]. Telecommunications Science, 2023, 39(6): 114-121.
[2]	Xue HAN. Damage toughness assessment method of power backbone communication network based on power big data [J]. Telecommunications Science, 2023, 39(5): 136-143.
[3]	Yudi SUN. A prediction model of massive 5G network users’ revisit behavior based on telecom big data [J]. Telecommunications Science, 2023, 39(2): 157-162.
[4]	Aihua LI, Xiaobo WU, Chao CHEN, Bin WEI, Yuanyuan SHI. Big data intelligent analysis technology for 5G network [J]. Telecommunications Science, 2022, 38(8): 129-139.
[5]	Gengsheng HUANG, Yuhong HUANG, Manxue GUO, Jianping ZHENG, Xin GE. Architecture of a new infrastructure for data service in DT era [J]. Telecommunications Science, 2022, 38(7): 138-145.
[6]	Panpan LI, Zhengxia XIE, Guangxue YUE, Xin LIU. Research progress and trends of deep learning based wireless communication receiving method [J]. Telecommunications Science, 2022, 38(2): 1-17.
[7]	Haibo ZHAO, Zhijun XIANG, Linsong XIAO. A big data framework for short-term power load forecasting using heterogenous data [J]. Telecommunications Science, 2022, 38(12): 103-111.
[8]	Zhiyong LIU, Zhongjiang HE, Yilong RUAN, Junfeng SHAN, Chao ZHANG. Big data security features and operation practices [J]. Telecommunications Science, 2021, 37(5): 160-169.
[9]	Zhaopin SU, Chaoyong SHEN, Guofu ZHANG, Feng YUE, Donghui HU. A robust audio steganography algorithm based on differential evolution [J]. Telecommunications Science, 2021, 37(11): 64-74.
[10]	Xiaojun LIU,Juan WU,Xiaoqing XU. Big data architecture analysis and data security fusion technology [J]. Telecommunications Science, 2020, 36(7): 146-155.
[11]	Yujiao LIU,Kunhuang SONG,Xiang WANG. Analysis of economic prosperity index based on big data of electricity [J]. Telecommunications Science, 2020, 36(6): 166-171.
[12]	Weihua CHENG,Xiaorong HE. Quality analysis method of Web browsing based on big data [J]. Telecommunications Science, 2020, 36(11): 174-181.
[13]	Xiaomin TAN,Ai FANG,Bing LIANG,Haojie YANG. Locating causes of abnormality of EPG experience based on decision tree [J]. Telecommunications Science, 2019, 35(8): 158-164.
[14]	Zelin LU,Qiangzhi LI. Evolutionary logic and operational mechanism of E-commerce platform [J]. Telecommunications Science, 2019, 35(7): 152-158.
[15]	Ming LIAO,Ming CHEN,Ji ZHOU,Xiaohua XIANG,Fang LI,Yefen JIAO. DNS log analysis system based on big data fusion algorithm [J]. Telecommunications Science, 2019, 35(5): 129-139.

Blog screening and mining based on temporal features and hybrid search in big data

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 17

Related Articles 15

Metrics

Recommended 0