基于主题模型的微博话题检测算法

doi:10.11959/j.issn.2096-109x.2016.00049

Abstract

Abstract:

Micro-blog data has the characteristic of real-time,volume,short-text,and noise-rich.So it is a challenge for the traditional topic detection technology.A novel micro-blog topic detection algorithm based on topic model was proposed.Firstly,the micro-blog data was expressed as text word matrix and word relation matrix.The topic word was extracted from the two vectors.Secondly,the topic model was obtained with clustering.Finally,the topic detection of micro-blog was obtained by clustering text and topic model.Experimental results show that the algorithm proposed can effectively detection the text topic,and with the best parameter group of precision,recall rate,F,and the value F is about 95%.

Key words: topic detection, topic model, text word matrix, word relation matrix

CLC Number:

TP391

Hua-jun HUANG,Jun-shan TAN,Jiao-hua QIN. Micro-blog topic detection algorithm based on topic model[J]. Chinese Journal of Network and Information Security, 2016, 2(5): 30-38.

Figures/Tables 12

References 13

[1]	王仲远, 程健鹏, 王海勋 ,等. 短文本理解研究[J]. 计算机研究与发展, 2016,53(2): 262-269.
	WANG Z Y , CHENG J P , WANG H X ,et al. Short text understand-ing:a survey[J]. Journal of Computer Research and Development, 2016,53(2): 262-269.
[2]	贺敏, 杜攀, 张谨 ,等. 基于动量模型的微博突发话题检测方法[J]. 计算机研究与发展, 2015,52(5): 1022-1028.
	HE M , DU P , ZHANG J et al. Microblog bursty topic detection method based on momentum model[J]. Journal of Computer Re-search and Development, 2015,52(5): 1022-1028.
[3]	刘全超, 黄河燕, 冯冲 . 基于多特征微博话题情感倾向性判定算法研究[J]. 中文信息学报, 2014,28(4): 123-131.
	LIU Q C , HUANG H Y , FENG C . Multi-feature based sentiment orientation identification algorithm for micro-blog topics[J]. Jour-nal of Chinese Information Processing, 2014,28(4): 123-131.
[4]	程俊霞, 李芝棠, 邹明光 ,等. 基于 SVM 过滤的微博新闻话题检测方法[J]. 通信学报, 2013,34(Z2): 74-78.
	CHEN J X , LI Z T , ZOU M G ,et al. Novel topic detection method for microblog based on SVM filtration[J]. Journal of Communi-cations, 2013,34(Z2): 74-78.
[5]	NIST. The 2003 topic detection and tracking task definition and evaluation plan[EB/OL]. .
[6]	周学广, 高飞, 孙燕 . 基于依存连接权 VSM 的子话题检测与跟踪方法[J]. 通信学报, 2013,34(8): 1-9.
	ZHOU X G , GAO F , SUN Y . Sub-topic detection and tracking based on dependency connection weights for vector space model[J]. Journal of Communications, 2013,34(8): 1-9.
[7]	付艳, 周明全, 王学松 ,等. 面向互联网新闻的在线事件检测[J]. 软件学报, 2010,21(Z): 363-372.
	FU Y , ZHOU M Q , WANG X S ,et al. On-line event detection from web news stream[J]. Journal of Software, 2010,21(Z): 363-372.
[8]	郝文宁, 冯波, 陈刚 ,等. 基于领域本体的文档向量空间模型构建[J]. 计算机应用研究, 2013,30(3): 764-767.
	HAO W N , FENG B , CHEN G ,et al. Document vector space model construction based on domain ontology[J]. Application Research of Computers, 2013,30(3): 764-767.
[9]	PHUVIPADAWAT S , MURATA T . breaking news detection and tracking in twitter[C]// 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology,Toronto. c2010: 120-123.
[10]	YANXIANG H , YE T , QIANG C ,et al. Summarizing microblogs on network hot topics[C]// 2011 International Conference on Internet Technology and Applications. c2011: 1-4.
[11]	谢娟英, 高红超 . 基于统计相夫性与k-means的区分基因子集选择算法[J]. 软件学报, 2014,25(9): 2050-2075.
	XIE J Y , GAO H C . Statistical correlation and k-means based dis-tinguishable gene subset selection algorithms[J]. Journal of Soft-ware, 2014,25(9): 2050-2075.
[12]	DAI X Y , CHEN Q C , WANG X L ,et al. Online topic detection and tracking of financial news based on hierarchical clustering[C]// 2010 International Conference on Machine Learning and Cybernetics,Qingdao. c2010: 3341-3346.
[13]	ZHAO W X , JIANG J , WENG J ,et al. Comparing twitter and traditional media using topic models[M]// Advances in Information Retrieval. Berlin:Heidelberg, 2011: 338-349.

Metrics

Recommended 0

No Suggested Reading articles found!

序号	话题	数量
1	#北京楼顶别墅主人#	386
2	#斯诺登棱镜事件#	131
3	#李天一案件#	356
4	#郑钧刘芸马尔代夫婚礼#	138
5	#姚贝娜vs林育群#	143
6	#七夕，一起看英仙座流星雨#	233
7	#光大证券乌龙事件#	480
8	#埃及清场#	153
9	#林丹vs李宗伟#	350
10	#台风尤特#	609
11	#泰国白龙王病逝#	160
12	#中国好外婆#	109
13	#昌都地震#	120
14	#新疆吐鲁番地表温度70℃火焰山烤鸡蛋生意红火#	87

参数话题	α=1，β=1，γ=1			α=3，β2，γ=5			α=5，β=2，γ=2
参数话题	P	R	F	P	R	F	P	R	F
1	24.55	56.48	34.22	100.00	56.21	71.97	98.97	100.00	99.48
2	11.27	100.00	20.26	95.42	95.41	95.42	99.20	94.66	96.87
3	14.64	36.51	20.90	97.79	87.07	92.12	98.72	87.08	92.54
4	8.78	56.52	15.20	100.00	92.75	96.24	100.00	92.75	96.24
5	35.46	69.93	47.06	97.10	93.71	95.37	99.24	91.61	95.27
6	88.78	37.33	52.57	75.57	99.57	85.92	82.01	97.85	89.24
7	41.22	99.79	58.34	26.42	56.04	35.91	99.35	95.83	97.56
8	6.08	35.29	10.37	84.70	94.12	89.16	96.57	92.16	94.31
9	29.05	73.71	41.68	20.82	60.57	30.99	99.69	94.57	97.07
10	60.23	75.86	67.15	29.37	49.09	36.75	98.22	99.67	98.94
11	9.47	68.75	16.64	95.18	98.75	96.93	95.18	98.75	96.93
12	5.51	44.95	9.82	100.00	52.29	68.67	99.03	94.50	96.71
13	6.12	39.17	10.69	96.46	90.83	93.56	100.00	94.17	97.00
14	6.64	58.62	11.94	100.00	88.51	93.90	100.00	87.36	93.25

话题	主题词
#北京楼顶别墅主人#	北京、别墅、楼顶、拆除、假山、张必清、顶盖、教授、山庄、小区
#斯诺登棱镜事件#	美国、斯诺登、棱镜
#李天一案件#	李天一、书法、酒吧、梦鸽、律师
#郑钧刘芸马尔代夫婚礼#	婚礼、马尔代夫、郑钧、刘芸
#姚贝娜vs林育群#	姚贝娜、声音、林育群
#七夕，一起看英仙座流星雨#	流星雨、流星、看到
#光大证券乌龙事件#	光大、证券、乌龙、交易、期货、市场、暴涨、股市
#埃及清场#	埃及、行动
#林丹vs李宗伟#	李宗伟、林丹、冠军、羽毛球、决赛、世界、对手、比赛
#台风尤特#	台风、尤特、强台风、登陆、中心
#泰国白龙王病逝#	龙王、泰国
#中国好外婆#	外婆、外孙、同志
#昌都地震#	地震、昌都、西藏
#新疆吐鲁番地表温度70℃火焰山烤鸡蛋生意红火#	鸡蛋、火焰山、地表

Micro-blog topic detection algorithm based on topic model

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 13

Related Articles 1

Metrics

Recommended 0