基于改进Transformer的社交媒体谣言检测

doi:10.11959/j.issn.2096-109x.2022042

Abstract

Abstract:

With the rapid development of the Internet, social media is increasingly integrated into all aspects of people’s daily life.Social media has gradually become a tool and even a platform for people to share opinions, insights, experiences and viewpoints.It is the main method for people to obtain and share information as well as express and exchange opinions.Currently, social media mainly includes social networking sites, Weibo, Twitter, blogs, forums, podcasts and so on.Due to the openness of social media, the user scale is large and the sources are complex and numerous, then all kinds of rumors and false information may be generated easily.Rumors on social media influence netizens’ understanding of events and shake the stability of society.Therefore, how to accurately and efficiently detect rumors has become an urgent problem to be solved.Existing Transformer based social media rumor detection models ignored the text location information.To effectively extract text location information and make full use of text potential information, a rumor detection model in social media was proposed and it was based on the enhanced Transformer.This model enhanced the traditional Transformer from two aspects of relative position and absolute position.It captured the direction information and distance information of the text using learnable relative position coding and mapped words from different positions to different feature spaces using absolute position coding.Experimental results show that, compared with the best benchmark model, the accuracy of the proposed model on Twitter15, Twitter16 and Weibo datasets is enhanced by 0.9%, 0.6% and 1.4%, respectively.Experimental results verify the effectiveness of the proposed location coding.And the enhanced Transformer based on location coding can significantly improve the effects of social media rumor detection.

Key words: rumor detection in social media, enhanced Transformer, position information

CLC Number:

TP391

Honghao ZHENG, Yinuo HAO, Hongtao YU, Shaomei LI, Yiteng WU. Rumor detection in social media based on eahanced Transformer[J]. Chinese Journal of Network and Information Security, 2022, 8(4): 168-174.

Figures/Tables 5

模型	准确率	F1值
模型	准确率	NR	FR	UR	TR
DTC	45.4%	73.3%	35.5%	41.5%	31.7%
SVM-TS	54.4%	79.6%	47.2%	48.3%	40.4%
DT-Rank	40.9%	50.1%	31.1%	47.3%	36.4%
GRU-2	64.6%	79.2%	57.4%	59.2%	60.8%
RvNN	72.3%	68.2%	75.8%	65.4%	82.1%
PPC	84.2%	81.1%	87.5%	79.0%	81.8%
GLAN	90.5%	92.4%	$91 . 7 %$	$92 . 7 %$	85.2%
Transformer-GAT	89.1%	93.8%	90.9%	82.8%	88.6%
TPE-GAT	$91 . 4 %$	$94 . 2 %$	91.2%	90.8%	$89 . 2 %$

模型	准确率	F1值
模型	准确率	NR	FR	UR	TR
DTC	46.5%	64.3%	39.3%	40.3%	41.9%
SVM-TS	57.4%	75.5%	42.0%	52.6%	57.1%
DT-Rank	41.4%	39.4%	27.3%	34.4%	63.0%
GRU-2	63.3%	77.2%	48.9%	59.3%	68.6%
RvNN	73.7%	66.2%	74.3%	70.8%	83.5%
PPC	86.3%	82.0%	$89 . 8 %$	83.7%	84.3%
GLAN	90.2%	$92 . 1 %$	86.9%	$96 . 8 %$	84.7%
Transformer-GAT	89.2%	90.5%	86.7%	85.7%	93.5%
TPE-GAT	$90 . 8 %$	90.4%	88.3%	92.2%	$92 . 1 %$

模型	类别	准确率	精准率	召回率	F1值
DTC	FR	83.1%	84.7%	81.5%	83.5%
	NR		81.5%	84.7%	83.0%
SVM-TS	FR	85.7%	83.9%	88.5%	86.1%
	NR		87.8%	83.0%	85.7%
DT-Rank	FR	73.2%	73.8%	71.5%	72.6%
	NR		72.6%	74.9%	73.7%
GRU-2	FR	91.0%	87.6%	95.6%	91.4%
	NR		$95 . 2 %$	86.4%	90.6%
PPC	FR	92.1%	89.6%	$96 . 2 %$	92.3%
	NR		94.9%	88.9%	91.8%
GLAN	FR	94.6%	94.3%	94.8%	94.5%
	NR		94.9%	94.3%	94.6%
Transformer-GAT	FR	93.4%	94.0%	92.8%	93.4%
	NR		92.8%	94.0%	93.4%
TPE-GAT	FR	$96 . 0 %$	$96 . 5 %$	95.5%	$96 . 0 %$
	NR		$95 . 2 %$	$97 . 0 %$	$96 . 1 %$

References 24

[1]	LIU Z Y , ZHANG L , TU C C ,et al. Statistical and semantic analysis of rumors in Chinese social media[J]. Scientia Sinica, 2015,45(12): 1536-1546.
[2]	MIKOLOV T , SUTSKEVER L , CHEN K ,et al. Distributed representations of words and phrases and their compositionality[J]. Advances in Neural Information Processing Systems, 2013,(26): 3111-3119.
[3]	RUCHANSKY N , SEO S , LIU Y . CSI:a hybrid deep model for fake news detection[J]. 2017:arXiv:1703.06959.
[4]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[5]	YUAN C Y , MA Q W , ZHOU W ,et al. Jointly embedding the local and global relations of heterogeneous graph for rumor detection[C]// Proceedings of 2019 IEEE International Conference on Data Mining (ICDM). 2019: 796-805.
[6]	琚心怡 . 基于深层双向 Transformer 编码器的早期谣言检测[J]. 信息通信, 2020,33(5): 17-22.
	QU X Y . Early rumor detection based on deep two-way Transformer encoder[J]. Information ＆ Communications, 2020,33(5): 17-22.
[7]	MA J , GAO W , WONG K F . Detect rumors in microblog posts using propagation structure via kernel learning[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 708-717.
[8]	MA J , GAO W , MITRA P ,et al. Detecting rumors from microblogs with recurrent neural networks[C]// Proceedings of International Joint Conference on Artificial Intelligence. 2016.
[9]	SHAW P , USZKOREIT J , VASWANI A . Self-attention with relative position representations[J]. arXiv preprint arXiv:1803.02155, 2018.
[10]	DEVLIN J , CHANG M W , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[11]	LIU Y H , OTT M , GOYAL N ,et al. RoBERTa:a robustly optimized BERT pretraining approach[J]. arXiv preprint arXiv:1907.11692, 2019.
[12]	YANG Z L , DAI Z H , YANG Y M ,et al. XLNet:generalized autoregressive pretraining for language understanding[J]. CoRR,2019,abs/1906.08237.
[13]	YAN H , DENG B C , LI X N ,et al. TENER:adapting transformer encoder for name entity recognition[J]. arXiv preprint arXiv:1911.04474, 2019.
[14]	DAI Z H , YANG Z L , YANG Y M ,et al. Transformer-XL:attentive language models beyond a fixed-length context[J]. arXiv preprint arXiv:1901.02860, 2019.
[15]	HE P C , LIU X D , GAO J F ,et al. DeBERTa:decoding-enhanced BERT with disentangled attention[J]. arXiv preprint arXiv:2006.03654, 2020.
[16]	KE G L , HE D , LIU T Y . Rethinking the positional encoding in language pre-training[J]. arXiv preprint arXiv:2006.15595, 2020.
[17]	WANG B Y , ZHAO D H , LIOMA C ,et al. Encoding word order in complex embeddings[J]. arXiv preprint arXiv:2006.15595, 2020.
[18]	LIU Y , WU Y F . Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks[C]// Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence. 2018.
[19]	MIKOLOV T , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[20]	SMITH L N , . Cyclical learning rates for training neural networks[C]// Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). 2017: 464-472.
[21]	CASTILLO C , MENDOZA M , POBLETE B . Information credibility on twitter[C]// Proceedings of the 20th International Conference on World Wide Web-WWW '11. 2011: 675-684.
[22]	MA J , GAO W , WEI Z Y ,et al. Detect rumors using time series of social context information on microblogging websites[C]// Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015: 1751-1754.
[23]	ZHAO Z , RESNICK P , MEI Q . Enquiring minds:early detection of rumors in social media from enquiry posts[C]// Proceedings of International World Wide Web Conferences Steering Committee. 2015.
[24]	MA J , GAO W , WONG K F . Rumor detection on twitter with tree-structured recursive neural networks[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 1980-1989.

Metrics

Recommended 0

No Suggested Reading articles found!

统计	Weibo	Twitter15	Twitter16
谣言文本	4 664	1 490	818
非谣言（NR）	2 351	374	205
虚假谣言（FR）	2 313	370	205
未经核实的谣言（UR）	0	374	203
真实谣言（TR）	0	372	205
用户	2 746 818	276 663	173 487

Rumor detection in social media based on eahanced Transformer

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 24

Related Articles 1

Metrics

Recommended 0