基于对比学习的社交媒体地理位置预测方法

doi:10.11959/j.issn.1000-0801.2023154

摘要/Abstract

摘要：

以往基于社交媒体文本的定位方法主要集中在将文本语义空间映射到地理位置空间，忽略了文本之间的语义相关性和地理位置之间的距离相关性。提出了一种新的无监督多层次对比学习框架，并设计了 3 个对比学习模块：语义学习模块、位置学习模块和跨层次学习模块。首先利用Transformer编码器获取文本的语义表示，以无监督的对比学习方式，聚拢位置相近文本之间的语义表示和地理表示，随后进行有监督训练，输出地理位置分类或回归结果。在4个数据集上与5个基线模型的对比实验结果表明，该框架有效地提升了社交媒体地理定位的准确性。

关键词: 社交媒体, 地理定位, 对比学习, 信息挖掘, Transformer

Abstract:

Previous work on social media text-based geolocation focused on mapping language semantic space to geospatial space, which ignores the semantic correlation between social media texts and the distance correlation between geographical locations.To take advantage of these correlations, mCLF, a new unsupervised multiple-level contrastive learning framework was proposed, three contrastive learning modules were designed: a semantic learning module, a location learning module, and a cross-learning module.Transformer encoder was used to obtain semantic representation of posts, utilizing unsupervised contrastive learning method to decrease the distance of semantic representations and location representations of posts with near locations, and then fine-tuned the model with supervised method for geographic location regression or classification outputs.Compared with five baseline methods, extensive experiments based on four datasets demonstrate the effectiveness of the proposed framework.

Key words: social media, geolocation, contrastive learning, information mining, Transformer

中图分类号:

TP391.1

徐永昌, 黄士多, 艾浩军. 基于对比学习的社交媒体地理位置预测方法[J]. 电信科学, 2023, 39(8): 58-68.

Yongchang XU, Shiduo HUANG, Haojun AI. A social media geolocation method based on comparative learning[J]. Telecommunications Science, 2023, 39(8): 58-68.

图/表 9

图1

表1

图2

表2

图3

表3

表4

表5

表6

参考文献 35

[1]	SAKAKI T , OKAZAKI M , MATSUO Y . Earthquake shakes Twitter users:real-time event detection by social sensors[C]// Proceedings of the 19th International Conference on World Wide Web. NewYork:ACM Press, 2010: 851-860.
[2]	KINSELLA S , MURDOCK V , O’HARE N . “I’m eating a sandwich in Glasgow”:modeling locations with tweets[C]// Proceedings of the 3rd International Workshop on Search and Mining User-generated Contents. NewYork:ACM Press, 2011: 61-68.
[3]	PAUL M J , DREDZE M . You are what your Tweet:analyzing twitter for public health[J]. Artificial Intelligence, 2011(38): 265-272.
[4]	DO T H , NGUYEN D M , TSILIGIANNI E ,et al. Multiview deep learning for predicting twitter users’ location[J]. arXiv preprint, 2017,arXiv:1712.08091.
[5]	WING B , BALDRIDGE J . Simple supervised document geolocation with geodesic grids[C]// Meeting of the Association for Computational Linguistics:Human Language Technologies. DBLP, 2012.
[6]	HAN B , COOK P , BALDWIN T . Geolocation prediction in social media data by finding location indicative words[J]. 24th International Conference on Computational Linguistics - Proceedings of COLING 2012:Technical Papers, 2012: 1045-1062.
[7]	RAHIMI A , COHN T , BALDWIN T . A neural model for user geolocation and lexical dialectology[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume2:Short Papers). Stroudsburg:Associationfor Computational Linguistics, 2017: 209-216.
[8]	JURGENS D . That’s what friends are for:inferring location in online social media platforms based on social relationships[J]. Proceedings of the International AAAI Conference on Web and Social Media, 2021,7(1): 273-282.
[9]	WANG F J , LU C T , QU Y Z ,et al. Collective geographical embedding for geolocating social network users[M]// Advances in Knowledge Discovery and Data Mining. Cham: Springer International Publishing, 2017: 599-611.
[10]	HUANG B X , CARLEY K . A hierarchical location prediction neural network for twitter user geolocation[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg:Associationfor Computational Linguistics, 2019: 4731-4741.
[11]	RAHIMI A , COHN T , BALDWIN T . Semi-supervised user geolocation via graph convolutional networks[J]. 2018: 2009-2019.DOI:10.18653/v1/P18-1187.
[12]	RAHIMI A , VU D , COHN T ,et al. Exploiting text and network context for geolocation of social media users[J]. 2015: 1362-1367.DOI:10.3115/v1/N15-1153.
[13]	SCALIA G , FRANCALANCI C , PERNICI B . CIME:context-aware geolocation of emergency-related posts[J]. GeoInformatica, 2022,26(1): 125-157.
[14]	ZHENG C , JIANG J Y , ZHOU Y C ,et al. Social media user geolocation via hybrid attention[C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2020: 1641-1644.
[15]	ALEXIS C , KIELA D . SentEval:an evaluation toolkit for universal sentence representations[J]. arXiv preprint, 2018,arXiv:1803.05449.
[16]	REIMERS N , GUREVYCH I . Sentence-BERT:sentence embeddings using Siamese BERT-networks[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg:Association for Computational Linguistics, 2019: 3980-3990.
[17]	GAO T Y , YAO X C , CHEN D Q . SimCSE:simple contrast ivelearning of sentence embeddings[C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Associationfor Computational Linguistics, 2021: 6894-6910.
[18]	GIORGI J , NITSKI O , WANG B ,et al. DeCLUTR:deep contrastive learning for unsupervised textual representations[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). Stroudsburg:Association for Computational Linguistics, 2021: 879-895.
[19]	CER D , YANG Y , KONG S Y ,et al. Universal sentence encod-er[J]. 2018: 169-174.DOI:10.18653/v1/D18-2029.2018.
[20]	HUANG J J , TANG D Y , ZHONG W J ,et al. WhiteningBERT:an easy unsupervised sentence embedding approach[C]// Proceedings of Findings of the Association for Computational Linguistics:EMNLP 2021. Stroudsburg:Associationfor Computational Linguistics, 2021: 238-244.
[21]	LI B H , ZHOU H , HE J X ,et al. On the sentence embeddings from pre-trained language models[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg:Association for Computational Linguistics, 2020: 9119-9130.
[22]	SU J , CAO J , LIU W ,et al. Whitening sentence representations for better semantics and faster retrieval[J]. arXiv preprint, 2021,arXiv:2103.15316.
[23]	CARLSSON F , GYLLENSTEN A C , GOGOULOU E ,et al. Semantic re-tuning with contrastive tension[C]// In 9th International Conference on Learning Representations,Austria:Virtual Event, 2021.
[24]	KIM T , YOO K M , LEE S G . Self-guided contrastive learning for BERT sentence representations[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). Stroudsburg:Association for Computational Linguistics, 2021: 2528-2540.
[25]	YAN Y M , LI R M , WANG S R ,et al. ConSERT:a contrastive framework for self-supervised sentence representation transfer[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1:Long Papers). Stroudsburg:Association for Computational Linguistics, 2021: 5065-5075.
[26]	MENG Y , XIONG C , BAJAJ P ,et al. COCO-LM:correcting and contrasting text sequences for language model pretraining[J]. arXiv preprint, 2021,arXiv:2102.08473.
[27]	WU Z , WANG S , GU J ,et al. CLEAR:contrastive learning for sentence representation[J]. arXiv preprint, 2020,arXiv:2012.15466.
[28]	HADSELLR , CHOPRA S , LECUN Y . Dimensionality reduction by learning an invariant mapping[C]// Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06). Piscataway:IEEE Press, 2006: 1735-1742.
[29]	EISENSTEIN J , O＇CONNOR B , SMITH N A ,et al. A latent variable model for geographic lexical variation[C]// Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing,EMNLP 2010,9-11 October 2010,MIT Stata Center,Massachusetts,USA,A meeting of SIGDAT,a Special Interest Group of the ACL. Stroudsburg:Association for Computational Linguistics, 2010.
[30]	CHAKRAVARTHI B R , GAMAN M , IONESCU R T ,et al. Findings of the VarDial evaluation campaign 2021[C]// Proceedings of the Eighth Workshop on NLP for Similar Languages,Varieties and Dialects,VarDial@EACL 2021,Kiyv,Ukraine, 2021: 1-11.
[31]	SCHERRER Y , LJUBE?I? N . Social media variety geolocation with geobert[C]// Proceedings of the Eighth Workshop on NLP for Similar Lan-guages,Varieties and Dialects. Stroudsburg:Association for Computational Linguistics, 2021.
[32]	RAHIMI A , BALDWIN T , COHN T . Continuous repre-sentation of location for geolocation and lexical dialectology using mixture density networks[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,EMNLP, 2017: 167-176.
[33]	RAHIMI A , VU D , COHN T ,et al. Exploiting text and network context for geolocation of social media users[C]// Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg:Association for Computational Linguistics, 2015: 1362-1367.
[34]	CHA M , GWON Y , KUNG H . Twitter geolocation and regional classification via sparse coding[J]. Proceedings of the International AAAI Conference on Web and Social Media, 2021,9(1): 582-585.
[35]	ROLLER S , SPERIOSU M , RALLAPALLI S ,et al. Supervised text-based geolocation using language models on an adaptive grid[J]. EMNLP-CoNLL2012- 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,Proceedings of the Conference, 2012: 1500-1510.

数据集名称	训练集	验证集	测试集	每个帖子Token数量
BCMS	353 953	38 013	4 189	13
DE-AT	318 487	2 416	31 515	69
CH	25 261	29 122	2 438	50
Geotext	302 064	37 758	37 758	12

方法	BCMS		DE-AT		CH		Geotext
方法	Median	Mean	Median	Mean	Median	Mean	Median	Mean
回归+受限
geoBERT	52.06	98.74	161.13	184.97	20.7	29.62	424.90	593.08
本文分类方法	37.35	88.63	160.11	182.53	20.25	28.55	410.01	566.52
回归+不受限
geoBERT	26.07	83.61	149.83	174.55	19.13	27.01	403.84	588.07
本文分类方法	20.88	81.33	149.28	173.57	18.76	26.74	379.75	561.26
分类+受限
MLP	—	—	—	—	—	—	389	844
MDN	—	—	—	—	—	—	412	865
LR	—	—	—	—	—	—	397	880
Sparse-Coding	—	—	—	—	—	—	425	581
geoBERT-cls	41.84	107.38	183.17	210.04	22.18	32.77	380.08	578.89
本文分类方法	15.75	89.62	181.09	208.62	18.80	30.59	326.24	557.84
分类+不受限
geoBERT-cls	15.00	90.96	170.25	197.40	21.61	30.77	356.43	572.77
本文分类方法	12.32	89.62	166.32	193.92	18.79	29.12	316.70	555.23

数据组	参考距离/km	欧氏距离
数据组	参考距离/km	mCLF	geoBERT
G.base-G.near	65.16	1.536 5	4.38
G.base-G.medium	222.05	1.808 5	4.69
G.base-G.far	347.56	1.954 2	3.07

方法	G.base		G.near		G.medium		G.far
方法	Median	Mean	Median	Mean	Median	Mean	Median	Mean
geoBERT	127.71	156.34	54.2	139.1	53.46	92.58	72.31	91.63
mCLF	107.54	147.63	48.38	126.31	37.53	81.14	71.39	88.8

实例	帖子内容	参考城市	误差距离/km
实例	帖子内容	参考城市	geoBERT	mCLF
1	Welcome to New York Antonio Cromartie, my friends from Ohio!	New York	413.03	157.37
2	@USER\_2ded78c9 and Miami still beat them in December, beat up and all.If Miami gets these upgrades (NT, LB, S, WR) it’ll be an even field.	Jacksonville	538.48	169.06
3	＆ @USER\_84b6a0b7 You should to come to FAMU.We’re having a forum on Texas Black Women and their various roles in society.	Tallahassee	999.68	351.09