基于改进Transformer的社交媒体谣言检测

doi:10.11959/j.issn.2096-109x.2022042

网络与信息安全学报 ›› 2022, Vol. 8 ›› Issue (4): 168-174.doi: 10.11959/j.issn.2096-109x.2022042

基于改进Transformer的社交媒体谣言检测

郑洪浩, 郝一诺, 于洪涛, 李邵梅, 吴翼腾

信息工程大学，河南郑州 450001

修回日期:2022-02-15 出版日期:2022-08-15 发布日期:2022-08-01
作者简介:郑洪浩（1992− ），男，山东济宁人，信息工程大学硕士生，主要研究方向为自然语言处理
郝一诺（1997− ），女，江苏徐州人，信息工程大学硕士生，主要研究方向为无线物理层安全、自然语言处理
于洪涛（1970− ），男，辽宁丹东人，博士，信息工程大学研究员、博士生导师，主要研究方向为网络大数据分析与处理
李邵梅（1982− ），女，湖北钟祥人，博士，信息工程大学副研究员，主要研究方向为计算机视觉
吴翼腾（1992− ），男，山东乐陵人，信息工程大学博士生，主要研究方向为人工智能安全
基金资助:
国家自然科学基金(61601513);郑州市协同创新重大专项(162/32410218)

Rumor detection in social media based on eahanced Transformer

Honghao ZHENG, Yinuo HAO, Hongtao YU, Shaomei LI, Yiteng WU

Information Engineering University, Zhengzhou 450001, China

Revised:2022-02-15 Online:2022-08-15 Published:2022-08-01
Supported by:
The National Natural Science Foundation of China(61601513);Major Collaborative Innovation Projects of Zhengzhou(162/32410218)

摘要/Abstract

摘要：

随着互联网的快速发展，社交媒体日益广泛而深刻地融入人们日常生活的各个方面。社交媒体逐渐成为人们彼此之间用来分享意见、见解、经验和观点的工具和平台，是人们获取分享信息、表达交流观点的主要途径。社交媒体在互联网的沃土上蓬勃发展，爆发出令人眩目的能量。由于社交媒体的开放性，用户规模庞大且来源复杂众多，容易产生各种各样的谣言虚假信息。社交媒体谣言左右着网民对事件的认识、动摇着社会的稳定。因此，如何准确高效地检测谣言成为当下亟待解决的问题。现有基于 Transformer的社交媒体谣言检测模型忽略了文本位置信息。为有效提取文本位置信息，充分利用文本潜在信息，提出了一种基于改进 Transformer 的社交媒体谣言检测模型。该模型从相对位置和绝对位置两方面对传统Transformer 进行改进：一方面采用可学习的相对位置编码捕捉文本的方向信息和距离信息；另一方面采用绝对位置编码将不同位置词语映射到不同特征空间。实验结果表明，与其他基准模型相比，所提模型在Twitter15、Twitter16和Weibo 3种数据集上的准确率分别提高了0.9%、0.6%和1.4%。实验结果验证了所提的位置编码改进有效，基于位置编码改进的Transformer模型可显著提升社交媒体谣言检测效果。

关键词: 社交媒体谣言检测, 改进Transformer, 位置信息

Abstract:

With the rapid development of the Internet, social media is increasingly integrated into all aspects of people’s daily life.Social media has gradually become a tool and even a platform for people to share opinions, insights, experiences and viewpoints.It is the main method for people to obtain and share information as well as express and exchange opinions.Currently, social media mainly includes social networking sites, Weibo, Twitter, blogs, forums, podcasts and so on.Due to the openness of social media, the user scale is large and the sources are complex and numerous, then all kinds of rumors and false information may be generated easily.Rumors on social media influence netizens’ understanding of events and shake the stability of society.Therefore, how to accurately and efficiently detect rumors has become an urgent problem to be solved.Existing Transformer based social media rumor detection models ignored the text location information.To effectively extract text location information and make full use of text potential information, a rumor detection model in social media was proposed and it was based on the enhanced Transformer.This model enhanced the traditional Transformer from two aspects of relative position and absolute position.It captured the direction information and distance information of the text using learnable relative position coding and mapped words from different positions to different feature spaces using absolute position coding.Experimental results show that, compared with the best benchmark model, the accuracy of the proposed model on Twitter15, Twitter16 and Weibo datasets is enhanced by 0.9%, 0.6% and 1.4%, respectively.Experimental results verify the effectiveness of the proposed location coding.And the enhanced Transformer based on location coding can significantly improve the effects of social media rumor detection.

Key words: rumor detection in social media, enhanced Transformer, position information

中图分类号:

TP391

郑洪浩, 郝一诺, 于洪涛, 李邵梅, 吴翼腾. 基于改进Transformer的社交媒体谣言检测[J]. 网络与信息安全学报, 2022, 8(4): 168-174.

Honghao ZHENG, Yinuo HAO, Hongtao YU, Shaomei LI, Yiteng WU. Rumor detection in social media based on eahanced Transformer[J]. Chinese Journal of Network and Information Security, 2022, 8(4): 168-174.

图/表 5

表1

表2

Twitter15数据集上的实验结果Table 2 Results on Twitter15 datasets"

模型	准确率	F1值
模型	准确率	NR	FR	UR	TR
DTC	45.4%	73.3%	35.5%	41.5%	31.7%
SVM-TS	54.4%	79.6%	47.2%	48.3%	40.4%
DT-Rank	40.9%	50.1%	31.1%	47.3%	36.4%
GRU-2	64.6%	79.2%	57.4%	59.2%	60.8%
RvNN	72.3%	68.2%	75.8%	65.4%	82.1%
PPC	84.2%	81.1%	87.5%	79.0%	81.8%
GLAN	90.5%	92.4%	$91 . 7 %$	$92 . 7 %$	85.2%
Transformer-GAT	89.1%	93.8%	90.9%	82.8%	88.6%
TPE-GAT	$91 . 4 %$	$94 . 2 %$	91.2%	90.8%	$89 . 2 %$

表2

表3

Twitter16数据集上的实验结果Table 3 Results on Twitter16 datasets"

模型	准确率	F1值
模型	准确率	NR	FR	UR	TR
DTC	46.5%	64.3%	39.3%	40.3%	41.9%
SVM-TS	57.4%	75.5%	42.0%	52.6%	57.1%
DT-Rank	41.4%	39.4%	27.3%	34.4%	63.0%
GRU-2	63.3%	77.2%	48.9%	59.3%	68.6%
RvNN	73.7%	66.2%	74.3%	70.8%	83.5%
PPC	86.3%	82.0%	$89 . 8 %$	83.7%	84.3%
GLAN	90.2%	$92 . 1 %$	86.9%	$96 . 8 %$	84.7%
Transformer-GAT	89.2%	90.5%	86.7%	85.7%	93.5%
TPE-GAT	$90 . 8 %$	90.4%	88.3%	92.2%	$92 . 1 %$

表3

表4

Weibo数据集上的实验结果Table 4 Results on Weibo datasets"

模型	类别	准确率	精准率	召回率	F1值
DTC	FR	83.1%	84.7%	81.5%	83.5%
	NR		81.5%	84.7%	83.0%
SVM-TS	FR	85.7%	83.9%	88.5%	86.1%
	NR		87.8%	83.0%	85.7%
DT-Rank	FR	73.2%	73.8%	71.5%	72.6%
	NR		72.6%	74.9%	73.7%
GRU-2	FR	91.0%	87.6%	95.6%	91.4%
	NR		$95 . 2 %$	86.4%	90.6%
PPC	FR	92.1%	89.6%	$96 . 2 %$	92.3%
	NR		94.9%	88.9%	91.8%
GLAN	FR	94.6%	94.3%	94.8%	94.5%
	NR		94.9%	94.3%	94.6%
Transformer-GAT	FR	93.4%	94.0%	92.8%	93.4%
	NR		92.8%	94.0%	93.4%
TPE-GAT	FR	$96 . 0 %$	$96 . 5 %$	95.5%	$96 . 0 %$
	NR		$95 . 2 %$	$97 . 0 %$	$96 . 1 %$

表4

表5

参考文献 24

[1]	LIU Z Y , ZHANG L , TU C C ,et al. Statistical and semantic analysis of rumors in Chinese social media[J]. Scientia Sinica, 2015,45(12): 1536-1546.
[2]	MIKOLOV T , SUTSKEVER L , CHEN K ,et al. Distributed representations of words and phrases and their compositionality[J]. Advances in Neural Information Processing Systems, 2013,(26): 3111-3119.
[3]	RUCHANSKY N , SEO S , LIU Y . CSI:a hybrid deep model for fake news detection[J]. 2017:arXiv:1703.06959.
[4]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[5]	YUAN C Y , MA Q W , ZHOU W ,et al. Jointly embedding the local and global relations of heterogeneous graph for rumor detection[C]// Proceedings of 2019 IEEE International Conference on Data Mining (ICDM). 2019: 796-805.
[6]	琚心怡 . 基于深层双向 Transformer 编码器的早期谣言检测[J]. 信息通信, 2020,33(5): 17-22.
	QU X Y . Early rumor detection based on deep two-way Transformer encoder[J]. Information ＆ Communications, 2020,33(5): 17-22.
[7]	MA J , GAO W , WONG K F . Detect rumors in microblog posts using propagation structure via kernel learning[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 708-717.
[8]	MA J , GAO W , MITRA P ,et al. Detecting rumors from microblogs with recurrent neural networks[C]// Proceedings of International Joint Conference on Artificial Intelligence. 2016.
[9]	SHAW P , USZKOREIT J , VASWANI A . Self-attention with relative position representations[J]. arXiv preprint arXiv:1803.02155, 2018.
[10]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[11]	LIU Y H , OTT M , GOYAL N ,et al. RoBERTa:a robustly optimized BERT pretraining approach[J]. arXiv preprint arXiv:1907.11692, 2019.
[12]	YANG Z L , DAI Z H , YANG Y M ,et al. XLNet:generalized autoregressive pretraining for language understanding[J]. CoRR,2019,abs/1906.08237.
[13]	YAN H , DENG B C , LI X N ,et al. TENER:adapting transformer encoder for name entity recognition[J]. arXiv preprint arXiv:1911.04474, 2019.
[14]	DAI Z H , YANG Z L , YANG Y M ,et al. Transformer-XL:attentive language models beyond a fixed-length context[J]. arXiv preprint arXiv:1901.02860, 2019.
[15]	HE P C , LIU X D , GAO J F ,et al. DeBERTa:decoding-enhanced BERT with disentangled attention[J]. arXiv preprint arXiv:2006.03654, 2020.
[16]	KE G L , HE D , LIU T Y . Rethinking the positional encoding in language pre-training[J]. arXiv preprint arXiv:2006.15595, 2020.
[17]	WANG B Y , ZHAO D H , LIOMA C ,et al. Encoding word order in complex embeddings[J]. arXiv preprint arXiv:2006.15595, 2020.
[18]	LIU Y , WU Y F . Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks[C]// Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence. 2018.
[19]	MIKOLOV T , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[20]	SMITH L N , . Cyclical learning rates for training neural networks[C]// Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). 2017: 464-472.
[21]	CASTILLO C , MENDOZA M , POBLETE B . Information credibility on twitter[C]// Proceedings of the 20th International Conference on World Wide Web-WWW '11. 2011: 675-684.
[22]	MA J , GAO W , WEI Z Y ,et al. Detect rumors using time series of social context information on microblogging websites[C]// Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015: 1751-1754.
[23]	ZHAO Z , RESNICK P , MEI Q . Enquiring minds:early detection of rumors in social media from enquiry posts[C]// Proceedings of International World Wide Web Conferences Steering Committee. 2015.
[24]	MA J , GAO W , WONG K F . Rumor detection on twitter with tree-structured recursive neural networks[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 1980-1989.

统计	Weibo	Twitter15	Twitter16
谣言文本	4 664	1 490	818
非谣言（NR）	2 351	374	205
虚假谣言（FR）	2 313	370	205
未经核实的谣言（UR）	0	374	203
真实谣言（TR）	0	372	205
用户	2 746 818	276 663	173 487

基于改进Transformer的社交媒体谣言检测

Rumor detection in social media based on eahanced Transformer

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 24

相关文章 1

Metrics

推荐阅读 0