基于RNN和主题模型的社交网络突发话题发现

doi:10.11959/j.issn.1000-436x.2018056

摘要/Abstract

摘要：

社交网络数据是稀疏和嘈杂的，并伴有大量的无意义话题。传统突发话题发现方法无法解决社交网络短文本稀疏性问题，并需要复杂的后处理过程。为了解决上述问题，提出一种基于循环神经网络（RNN,recurrent neural network）和主题模型的突发话题发现（RTM-SBTD）方法。首先，综合RNN和逆序文档频率（IDF,inverse document frequency）构建权重先验来学习词的关系，同时通过构建词对解决短文本稀疏性问题。其次，模型中引入针板先验（spike and slab）来解耦突发话题分布的稀疏和平滑。最后，引入词的突发性来区分建模普通话题和突发话题，实现突发话题自动发现。实验结果表明与现有的主流突发话题发现方法相比，所提 RTM-SBTD 方法在多种评价指标上优于对比算法。

关键词: 社交网络, 突发话题发现, 主题模型, 循环神经网络

Abstract:

The data is noisy and diverse,with a large number of meaningless topics in social network.The traditional method of bursty topic discovery cannot solve the sparseness problem in social network,and require complicated post-processing.In order to tackle this problem,a bursty topic discovery method based on recurrent neural network and topic model was proposed.Firstly,the weight prior based on RNN and IDF were constructed to learn the relationship between words.At the same time,the word pairs were constructed to solve the sparseness problem.Secondly,the “spike and slab” prior was introduced to decouple the sparsity and smoothness of the bursty topic distribution.Finally,the burstiness of words were leveraged to model the bursty topic and the common topic,and automatically discover the bursty topics.To evaluate the effectiveness of proposed method,the various experiments were conducted.Both qualitative and quantitative evaluations demonstrate that the proposed RTM-SBTD method outperforms favorably against several state-of-the-art methods.

Key words: social network, bursty topic discovery, topic model, RNN

中图分类号:

TP393

石磊,杜军平,梁美玉. 基于RNN和主题模型的社交网络突发话题发现[J]. 通信学报, 2018, 39(4): 189-198.

Lei SHI,Junping DU,Meiyu LIANG. Social network bursty topic discovery based on RNN and topic model[J]. Journal on Communications, 2018, 39(4): 189-198.

图/表 10

图1

图2

图3

表1

符号及其描述"

符号	意义
D、N_p	短文本数据集、词对数量
K、 p	主题数量、词对集合
φ_c、θ_k	常态词分布、突发话题分布
$b_{z} 、 μ_{w}^{t}$	主题选择器、词对的突发概率
Z、π	主题分配、二值开关变量
$α 、 \bar{α}$	平滑先验、弱平滑先验
γ₀、γ1	超参数
A_z	聚焦主题集合
I[?]	示性函数

表1

表2

图4

图5

表3

表4

表5

参考文献 28

[1]	方滨兴, 贾焰, 韩毅 . 社交网络分析核心科学问题、研究现状及未来展望[J]. 中国科学院院刊, 2015(2): 187-199.
	FANG B X , JIA Y , HAN Y . Social network analysis-key research problems,related work,and future prospects[J]. Bulletin of Chinese Academy of Sciences, 2015(2): 187-199.
[2]	贾焰, 甘亮, 李爱平 . 社交网络智慧搜索研究进展与发展趋势[J]. 通信学报, 2015,36(12): 9-16.
	JIA Y , GAN L , LI A P . Research progress and development trend of online social network smart search[J]. Journal on Communications, 2015,36(12): 9-16.
[3]	王晓阳, 郑骁庆, 肖仰华 . 智慧搜索中的实体与关联关系建模与挖掘[J]. 通信学报, 2015,36(12): 17-27.
	WANG X Y , ZHENG X Q , XIAO Y H . Entity-relation modeling and discovery for smart search[J]. Journal on Communications, 2015,36(12): 17-27.
[4]	黄河燕 . 在线社交网络的可视化分析[J]. 中国科学院院刊, 2015,(2): 229-237.
	HUANG H Y . Visual analysis of online social networks[J]. Bulletin of Chinese Academy of Sciences, 2015,(2): 229-237.
[5]	唐杰, 陈文光 . 面向大社交数据的深度分析与挖掘[J]. 科学通报, 2015,60(5): 509-519.
	TANG J , CHEN W G . Deep analytics and mining for big social data[J]. Chinese Science Bulletin, 2015,60(5): 509-519.
[6]	WANG Y , LIU J , HUANG Y . Using hashtag graph-based topic model to connect semantically-related words without co-occurrence in microblogs[J]. IEEE Transactions on Knowledge and Data Engineering, 2016,28(7): 1919-1933.
[7]	MCMINN A J , JOSE J M . ,Real-time entity-based event detection for Twitter[C]// International Conference of the Cross-Language Evaluation Forum for European Languages. 2015: 65-77.
[8]	CHENG X , YAN X , LAN Y . BTM:topic modeling over short texts[J]. IEEE Transactions on Knowledge ＆ Data Engineering, 2014,26(12): 2928-2941.
[9]	QUAN X , KIT C , GE Y . Short and sparse text topic modeling via self-aggregation[C]// International Conference on Artificial Intelligence. 2015: 2270-2276.
[10]	HOFFMAN M D , BLEI D M , BACH F . Online learning for latent dirichlet allocation[C]// International Conference on Neural Information Processing Systems. 2010: 856-864.
[11]	STILO G , VELARDI P . Efficient temporal mining of micro-blog texts and its application to event discovery[J]. Data Mining and Knowledge Discovery, 2016,30(2): 372-402.
[12]	YAN X , GUO J , LAN Y . A probabilistic model for bursty topic discovery in microblogs[C]// Twenty-Ninth AAAI Conference on Artificial Intelligence. 2015: 353-359.
[13]	LAU J H , COLLIER N , BALDWIN T . On-line trend analysis with topic models:# twitter trends detection topic model online[C]// COLING. 2012: 1519-1534.
[14]	CAO Z , LI S , LIU Y . A novel neural topic model and its supervised extension[C]// 29th AAAI Conference on Artificial Intelligence. 2015: 2210-2216.
[15]	XIE W , ZHU F , JIANG J . Topicsketch:real-time bursty topic detection from twitter[J]. IEEE Transactions on Knowledge and Data Engineering, 2016,28(8): 2216-2229.
[16]	GAO Y , WEN D , CHEN NS . A novel contextual topic model for multi-document summarization[J]. Expert Systems with Applications, 2015,42(3): 1340-1352.
[17]	PETROVI? S , OSBORNE M , LAVRENKO V . Streaming first story detection with application to twitter[C]// Human Language Technologies:The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2010: 181-189.
[18]	BECKER H , NAAMAN M , GRAVANO L . Beyond trending topics:real-world event identification on twitter[J]. ICWSM, 2011(11): 438-441.
[19]	LI C , SUN A , DATTA A . Twevent:segment-based event detection from tweets[C]// ACM International Conference on Information and Knowledge Management. 2012: 155-164.
[20]	HUANG J , PENG M , WANG H . A probabilistic method for emerging topic tracking in Microblog stream[J]. World Wide Web-Internet ＆Web Information Systems, 2016,20(2): 1-26.
[21]	SUTSKEVER I , MARTENS J , HINTON G E . Generating text with recurrent neural networks[C]// International Conference on Machine Learning. 2011: 1017-1024.
[22]	AMIRI H , DAUMé III H . Short text representation for detecting churn in microblogs[C]// AAAI. 2016: 2566-2572.
[23]	XIA Y , TANG N , HUSSAIN A . Discriminative bi-term topic model for headline-based social news clustering[C]// FLAIRS Conference. 2015: 311-316.
[24]	LU H , XIE L Y , KANG N . Don't forget the quantifiable relationship between words:using recurrent neural network for short text topic discovery[C]// AAAI. 2017: 1192-1198.
[25]	WANG C , BLEI D M . Decoupling sparsity and smoothness in the discrete hierarchical dirichlet process[C]// Advances in neural information processing systems. 2009: 1982-1989.
[26]	LIN T , TIAN W , MEI Q . The dual-sparse topic model:mining focused topics and focused terms in short text[C]// International Conference on World Wide Web. 2014: 539-550.
[27]	GRIFFITHS T L , STEYVERS M . Finding scientific topics[J]. The National Academy of Sciences, 2004,101(1): 5228-5235.
[28]	NEWMAN D , LAU J H , GRIESER K . Automatic evaluation of topic coherence[C]// Human Language Technologies:The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2010: 100-108.

方法	P@10	P@30	P@50
RTM-SBTD	0.803	0.822	0.829
OnlineLDA	0.228	0.213	0.186
BBTM	0.720	0.732	0.724
SATM	0.563	0.478	0.449
Twevent	0.711	0.725	0.689

算法	前10个词
RTM-SBTD	飞机、乘客、马航、失联、MH370、遇难、客机、平安、声明、祈祷
BBTM	客机、击落、飞机、坠毁、马航、服务、俄罗斯、乘客、中国、平安
Twevent	马来西亚、乌克兰、恐怖、贵宾厅、航班、天气、公司、护照、艾滋病、绝望
OnlineLDA	北京、入境处、乘务员、MH370、护照、消息、日本、马航、报道、事件
SATM	祈福、飞机、安息、手机、购物、旅游、华为、北京、马航、机场

算法	K=5	K=10	K=15	K=20	K=25	K=30
RTM-SBTD	0.289	0.412	0.489	0.520	0.553	0.559
OnlineLDA	0.264	0.363	0.422	0.461	0.475	0.486
BBTM	0.271	0.387	0.457	0.478	0.489	0.515
Twevent	0.173	0.184	0.187	0.226	0.213	0.209
SATM	0.232	0.359	0.438	0.459	0.467	0.471

算法	K=5	K=10	K=15	K=20	K=25	K=30
RTM-SBTD	0.109	0.092	0.089	0.088	0.085	0.084
OnlineLDA	0.111	0.097	0.096	0.094	0.093	0.094
BBTM	0.110	0.094	0.093	0.090	0.088	0.087
Twevent	0.122	0.124	0.125	0.126	0.128	0.131
SATM	0.113	0.099	0.097	0.095	0.094	0.095