基于子语义空间的挖掘短文本策略方法

doi:10.11959/j.issn.1000-0801.2020061

摘要/Abstract

摘要：

为解决精准识别短文本数据的问题，提出一种基于子语义空间的短文本策略挖掘方法。该方法首先采用语义空间技术，解决短文本在分析过程中存在的“词汇鸿沟”与“数据稀疏”问题；然后基于聚类算法将语义空间划分为多个子语义空间，在各子语义空间并行挖掘关联规则，提高了策略生成的效率与质量；最后利用二叉树进行策略归并，生成最简策略集。实验证明，与传统的分类模型相比，该方案生成的策略集在误报率为6.5%的情况下，准确率可达88%。在违规短信的发现处理中，使用该技术挖掘的策略集，覆盖能力强、准确率高，具有很强的实用性。

关键词: 子语义空间, 策略提取, 短文本, 关联规则挖掘, 聚类

Abstract:

To solve the problem of identifying short text data accurately,a method of short text strategy mining based on sub-semantic space was proposed.Firstly,semantic space technology was used to solve the problem of “vocabularygap” and “data sparseness” in short text analysis.Then,based on clustering algorithm,the semantic space was divided into several sub-semantic spaces,and association rules were mined in the sub-semantic space,which improved the efficiency and quality of strategy generation.Finally,binary tree was used to merge strategies and generate the simplest strategy set.Experiments show that compared with the traditional classification model,the accuracy rate of the strategy set generated by the proposed scheme can achieve 85% when the false positive rate is 6.5%.In the processing of illegal short messages,using this technology to mine potential policy sets has strong coverage ability,high accuracy and strong practicability.

Key words: sub-semantic space, strategy extraction, short text, association rule mining, clustering

中图分类号:

TP391.4

孙洋,粟栗,张星,王峰生,杜海涛. 基于子语义空间的挖掘短文本策略方法[J]. 电信科学, 2020, 36(3): 83-94.

Yang SUN,Li SU,Xing ZHANG,Fengsheng WANG,Haitao DU. Method of short text strategy mining based on sub-semantic space[J]. Telecommunications Science, 2020, 36(3): 83-94.

图/表 14

图1

表1

表2

表3

图2

表4

表5

图3

表6

表7

图4

表8

表9

表10

参考文献 11

[1]	YIH W , GOODMAN J , CARVALHO V R . Finding advertising keywords on Web pages[C]// Proceedings of the 15th International Conference on World Wide Web. New York:ACM Press, 2006: 213-222.
[2]	KUHN R , DE MORI R . A cache-based natural language model for speech recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990,12(6): 570-583.
[3]	MIHALCEA R , TARAU P . TextRank:bringing order into texts[Z]. 2004.
[4]	王庆, 陈泽亚, 郭静 ,等. 基于词共现矩阵的项目关键词词库和关键词语义网络[J]. 计算机应用, 2015,35(6): 1649-1653.
	WANG Q , CHEN Z Y , GUO J ,et al. Project keyword lexicon and keyword semantic network based on word co-occurrence matrix[J]. Journal of Computer Applications, 2015,35(6): 1649-1653.
[5]	董振东, 董强, 郝长伶 . 知网的理论发现[J]. 中文信息学报, 2007,21(4): 3-9.
	DONG Z D , DONG Q , HAO C L . Theoretical findings of HowNet[J]. Journal of Chinese Information Processing, 2007,21(4): 3-9.
[6]	BENGIO Y , DUCHARME R , VINCENT P ,et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003(3): 1137-1155.
[7]	MIKOLOV T , CHEN K , CORRADO G ,et al. Efficient estimation of word representations in vector space[J]. Computer Science, 2013.
[8]	BENGIO Y , DUCHARME R , WINCENT P . Neural probabilistic language model neural probabilistic language model[Z]. 2003.
[9]	HANM J W , KAMBER M , PEI J . 数据挖掘概念与技术[M]. 范明,孟小峰,译 .北京: 机械工业出版社, 2012: 157-179.
	HANM J W , KAMBER M , PEI J . Data mining concepts and techniques[M]. Translated by FAN M,MENG X F, Beijing: Machinery Industry PressPress, 2012: 157-179.
[10]	朱龙珠, 徐宏, 刘莉莉 . 基于深度学习的 95598 重大服务事件识别研究[J]. 电力信息与通信技术, 2018,16(11): 19-23.
	ZHU L Z , XU H , LIU L L . Research on recognition of 95598 significant service events based on deep learning[J]. Electric Power Information and Communication Technology, 2018,16(11): 19-23.
[11]	陈涛, 鲁萌, 陈彦名 . 运营商大数据技术应用研究[J]. 电信科学, 2017,33(1): 130-134.
	CHEN T , LU M , CHEN Y M . Research on operators’ big data technologies and applications[J]. Telecommunications Science, 2017,33(1): 130-134.

类别	词汇
推广类	好礼\|豪礼、大放送、赔钱\|亏本\|贱价、火热\|火爆、喜悦、赔钱\|贱卖\|赔本\|亏本、相思\|思念、传达\|传送\|传递、便民、主流、粉丝
欺诈类	房租\|租金\|房费、卡上\|卡里\|卡内\|账户\|账上\|账号、房东、幸运、评选\|选中\|抽选\|抽中\|中选、旺肖\|平肖\|特肖、给料\|发料、办证、刻章、发票、积分
违法类	遇难、法轮、谎言、机缘、彷徨、劫难\|灾难、佛法、罪恶\|罪行、镇压、功学、保佑、中共\|共产党\|中央
涉黑类	流连忘返\|乐不思蜀、排列、输钱、百家乐\|六合采\|六合彩\|赌球\|打麻将、随时随地\|足不出户\|不出门、赌博\|博弈、间断、点子\|运到\|运气、先知、赌王、分金、储直\|充值、http\|www.\|.com\|.c0m\|.cn\|.cc\|.hk\|.net\|.apk\|m.\|.lt

策略词	特征ID	支持度
百家乐\|六合采\|六合彩\|赌球\|打麻将	T1	60
赌博\|博弈	T2	60
http\|www.\|.com\|.c0m\|.cn\|.cc\|.hk\|.net\|.apk\|m.\|.lt	T3	60
点子\|运到\|运气	T4	40
分金	T5	20
排列	T6	20
输钱	T7	20
先知	T8	20
随时随地\|足不出户\|不出门	T9	10

短信ID	策略词表示	特征ID表示	支持度
M1	百家乐\|六合采\|六合彩\|赌球\|打麻将,赌博\|博弈,http\|www.\|.com\|.c0m\|.cn\|.cc\|.hk\|.net\| .apk\|m.\|.lt,随时随地\|足不出户\|不出门	T1,T2,T3,T4	40
M2	百家乐\|六合采\|六合彩\|赌球\|打麻将,分金,排列	T1,T5,T6	10
M3	百家乐\|六合采\|六合彩\|赌球\|打麻将,http\|www.\|.com\|.c0m\|.cn\|.cc\|.hk\|.net\|.apk\|m.\|.lt,分金,排列	T1,T3,T5,T6	10
M4	赌博\|博弈,输钱,先知,随时随地\|足不出户\|不出门	T2,T7,T8,T9	10
M5	赌博\|博弈,http\|www.\|.com\|.c0m\|.cn\|.cc\|.hk\|.net\|.apk\|m.\|.lt ,输钱,先知	T2,T3,T7,T8	10

ITEM_ID	模式基	条件FP_TREE	关联规则
T9	{(T2 T7 T8:10)}	<T2:10 ,T7:10 ,T8:10>	T2T9:10,T7T9:10,T8T9:10,T2T7T9:10,T2T8T9:10,T2T7T8 T9:10
T8	{ (T2 T7:10)	<T2:10,T7:10>	T2T8:20,T7T8:10,T2T7T8:20,T3T8:10,T2T3T7T8:10
	(T2 T3 T7:10)}	<T2:10,T3:10,T7:10>
T7	{(T2:20)}	< T2:20 >	T2T7:20
T6	{(T1 T2 T5:60)	<T1:60,T2:60 ,T5:60>	T1T6:70,T2T6:70,T5T6:70,T1T2T6:70,T1T5T6:70,T2T5T6:70,T1T2T5T6:70,T1T2T3T6:10,T2T3T5:10,T1T3T5:10,T1T2T3T5T6:10
	(T1 T2 T3 T5:10)}	<T1:10,T2:10,T3:10,T5:10>
T5	{(T1 T2:10)	<T1:20,T2:20,T3:10>	T1T5:20,T2T5:20,T3T5:10,T1T2T5:20,T1T3T5:10,T2T3T5:10,T1T2T3T5:10
	(T1 T2 T3:10)}
T4	{(T1 T2 T3:40)}	<T1:40,T2:40,T3:40>	T1T4:40,T2T4:40,T3T4:40,T1T2T4:40,T1T3T4:40,T2T3T4:40,T1T2T3T4:40
T3	{(T1 T2:40)	<T1:50,T2:50><T2:20>	T1T3:50,T2T3:70,T1T2T3:50
	(T1 T2:10)(T2:20)}
T2	{(T1:60)}	<T1:60>	T2T1:60

策略ID	排序前	排序后
0001	T1＆T6	T6＆T1
0002	T2＆T6	T6＆T2
0003	T1＆T2＆T6	T6＆T1＆T2
0004	T1＆T5＆T6	T6＆T1＆T5
0005	T1＆T2＆T5＆T6	T6＆T1＆T2＆T5