特征词选择与相似度融合的微博话题发现方法

doi:10.11959/j.issn.1000-0801.2017258

Abstract

Abstract:

Some words existing in micro-blog short text have a bad effect on the accuracy of text similarity calculation,further affecting the quality of topic discovery.And these words are the same in shape or semantic meaning,but remote from the topic.A novel method of feature words selection based on micro-blog short text content and structured information was proposed,which could effectively choose some important feature words from the text.Moreover,in computing the similarity between texts,an improvement on computing the similarity between the text and the topic was made.Finally,the methods were combined together and applied to discover micro-blog topics.Experimental results show that the new method of topic discovery can effectively reduce the average missing rate and false detection rate,and improve the quality of topic discovery.

Key words: micro-blog, feature word, selection, similarity, topic discovery

CLC Number:

TP391

Hongyang CHEN,Linlin WANG,Yingsheng CHEN,Jiangkun LU,Xue ZUO. A method of micro-blog topic discovery based on feature words selection and text similarity[J]. Telecommunications Science, 2017, 33(10): 134-140.

Figures/Tables 3

References 10

[11]	黄贤英, 陈红阳, 刘英涛 . 短文本相似度研究及其在微博话题检测中的应用[J]. 计算机工程与设计, 2015,36(11): 3128-3133.
	HUANG X Y , CHEN H Y , LIU Y T . Research on micro-blog short text similarity and its application in micro-blog topic detection[J]. Computer Engineering and Design, 2015,36(11): 3128-3133.
[1]	丁兆云, 贾焰, 周斌 . 微博数据挖掘研究综述[J]. 计算机研究与发展, 2016(7): 98-105.
	DING Z Y , JIA Y , ZHOU B . Survey of data ming for micro-blogs[J]. Journal of Computer Research and Development, 2016(7): 98-105.
[2]	琚春华, 鲍福光, 戴俊彦 . 一种融入公众情感投入分析的微博话题发现与细分方法[J]. 电信科学, 2014,14(12): 227-231.
	JU C H , BAO F G , DAI J Y . Discovery and segmentation method in micro-blog topics based on public emotional engagement analysis[J]. Telecommunications Science, 2014,14(12): 227-231.
[3]	刘彦伟 . 微博话题追踪系统的研究与实现[D]. 北京:北京交通大学, 2013.
	LIU Y W . Research and implementation of micro-blog topic tracking system[D]. Beijing:Beijing Jiaotong University, 2013.
[4]	丁荩 . 微博热点发现技术的研究与实现[D]. 武汉:华中科技大学, 2012.
	DING J . Research and implementation of micro-blog hot topic detection[D]. Wuhan:Huazhong University of Science and Technology, 2012.
[5]	姜芳, 李国和, 岳翔 . 基于语义的文档特征提取研究方法[J]. 计算机科学, 2016,43(2): 254-258.
	JIANG F , LI G H , YUE X . Semantic-based feature extraction method for document[J]. Computer Science, 2016,43(2): 254-258.
[6]	黄贤英, 陈红阳, 刘英涛 ,等. 一种新的微博短文本特征词选择算法[J]. 计算机工程与科学, 2015,37(9): 1762-1767.
	HUANG X Y , CHEN H Y , LIU Y T ,et al. A novel method of feature selection on micro-blog short text[J]. Computer Engineering and Science, 2015,37(9): 1762-1767.
[7]	HUANG S , YANG Y , LI H ,et al. Topic detection from microblog based on text clustering and topic model analysis[C]// 2014 Asia-Pacific Services Computing Conference (APSCC 2014),Dec 4-6,2014,Fuzhou,China. New Jersey:IEEE Press, 2014: 88-92.
[8]	胡长龙, 唐晋韬, 王挺 . 中文微博的hashtag话题相关性分析[J]. 计算机科学, 2013,40(11A): 235-245.
	HU C L , TANG J T , WANG T . Topic relevance analysis of hashtags in Chinese micro-blogging environment[J]. Computer Science, 2013,40(11A): 235-245.
[9]	ZHENG J , LI Y . A hot topic detection method for chinese microblog based on topic words[C]// 2014 2nd International Conference on International Conference on Information Technology and Electronic Commerce,Dec 20-21,2014,Dalian,China. New Jersey:IEEE Press, 2014.
[10]	孙建旺, 吕学强, 张雷瀚 . 基于语义和最大匹配度的短文本分类研究[J]. 计算机工程与设计, 2013,34(10): 3613-3618.
	SUN J W , LV X Q , ZHANG L H . Short text classification based on semantics and maximum matching degree[J]. Computer Engineering and Design, 2013,34(10): 3613-3618.

Metrics

Recommended 0

No Suggested Reading articles found!

微博话题	原始的微博文本内容	经原有特征词算法选取后的微博文本内容特征词	经改进特征词算法选取后的微博文本特征词
“昌都地震”中的微博文本1	【西藏昌都地震已造成 21 间房屋倒塌】中新网拉萨8月12日电，据西藏昌都地区左贡县应急办消息：截至12日中午13时30分，发生于清晨5时23分的6.1级地震，已造成当地21间房屋倒塌。（分享自 @凤凰网） http://t.cn/zQHRtJo	西藏，地震，造成，房屋，倒塌，拉萨，月，日，电，西藏，地区，应急，办，消息，午时，清晨，时分，级，地震，造成，当地，房屋，倒塌，凤凰，网	西藏，昌都，地震，造成，房屋，倒塌，拉萨，西藏，昌都，地区，左贡县，应急，办，消息，中午，发生，清晨，时分，级，地震，造成，当地，房屋，倒塌，通讯，交通，条件，落后，已经，派出，多股，救援，力量，前往，地震，震中
“昌都地震”中的微博文本2	【西藏昌都地震暂无人员伤亡报告】西藏“左贡县与芒康县”交界处，8月12日发生地震后，西藏军区迅速“就近”派出部队官兵、医护人员及民兵350多人，赶赴灾区，展开救援	西藏，地震，报告，西藏，发生，地震，西藏，军区，迅速，就近，派出，部队，官兵，赶赴，灾区，展开，救援	西藏，昌都，地震，无，人员，伤亡，报告，西藏，发生，地震，西藏，军区，迅速，就近，派出，部队，官兵，医护，人员，民兵，赶赴，灾区，展开，救援

微博话题发现的方法	漏检率	误检率	归一化错误识别代价
参考文献[11]中的微博话题发现方法	0.154 8	0.020 4	0.254 8
基于原有特征词选择与相似度融合的微博话题发现方法	0.128 6	0.014 3	0.198 8
本文提出的微博话题发现方法	0.112 8	0.012 4	0.173 6

[1]	Jingjie SHEN, Guangqiu LI, Yancui LUO, Huizhi LIU. Physical layer security of full-duplex relay-assisted D2D networks under outdated CSI [J]. Telecommunications Science, 2023, 39(3): 89-99.
[2]	Yifan DING, Guangqiu LI, Hui LI. Physical layer security of NOMA-D2D cooperative wireless system [J]. Telecommunications Science, 2022, 38(9): 83-94.
[3]	Yu DING, Chenkai LI, Huimei HAN, Weidang LU, Yuanhong REN, Yuan GAO, Jiang CAO. Multi-agent heterogeneous network selection method based on 5G UAV communication [J]. Telecommunications Science, 2022, 38(8): 28-36.
[4]	Liang LI, Xinjie YANG. A multi-hop D2D and social awareness based novel relay selection algorithm [J]. Telecommunications Science, 2022, 38(8): 101-110.
[5]	Xin LI, Jun SUN. Value-difference learning based mMTC devices access algorithm in multi-cell network [J]. Telecommunications Science, 2022, 38(6): 82-90.
[6]	Yue WANG, Fangmin XU, Haiyan CAO. Pilot interference mitigation for TDD cell-free massive MIMO systems [J]. Telecommunications Science, 2022, 38(5): 87-94.
[7]	Huajian REN, Xiulan HAO, Wenjing XU. Deep learning Chinese input method with incremental vocabulary selection [J]. Telecommunications Science, 2022, 38(12): 56-64.
[8]	Ying CHENG, Guangqiu LI, Jingjie SHEN, Liang WEI. Physical layer security performance of MF relay selection systems [J]. Telecommunications Science, 2021, 37(9): 95-104.
[9]	Peichang ZHANG, Wannian AN, Shida ZHONG, Lei HUANG, Chunsheng ZHUANG, Wei ZHANG. A novel low-complexity two-tier group antenna selection algorithm [J]. Telecommunications Science, 2021, 37(7): 67-76.
[10]	Guoqing LIU, Xingqi WANG, Dan WEI, Jinglong FANG, Yanli SHAO. Feature selection method for software defect number prediction based on maximum information coefficient [J]. Telecommunications Science, 2021, 37(5): 133-147.
[11]	Songbai LIANG, Yingwu GUO, Xinwei LI. Network quality and user perception improvement of high-speed rail scene [J]. Telecommunications Science, 2021, 37(3): 162-170.
[12]	Yunhe YU, Jun SUN. Research on resource allocation algorithm of centralized and distributed Q-learning in machine communication [J]. Telecommunications Science, 2021, 37(11): 41-50.
[13]	Jingcheng CAO, Jidong ZHANG, Guojie SHI. A method for improving recall rate of similar image retrieval by using edge enhancement technology [J]. Telecommunications Science, 2021, 37(1): 76-84.
[14]	Feng LIN,Liujing XU,Xiaohua CHEN,Weiqiang QI,Ke CHEN,Tiantian ZHU. Method of Webshell detection based on multi-view feature fusion [J]. Telecommunications Science, 2020, 36(6): 125-132.
[15]	Hui QIAN,Guangqiu LI,Yifan DING. Physical layer security performance of SWIPT system in the presence of randomly located eavesdroppers [J]. Telecommunications Science, 2020, 36(5): 65-72.

A method of micro-blog topic discovery based on feature words selection and text similarity

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 3

References 10

Related Articles 15

Metrics

Recommended 0