基于DPI数据挖掘实现URL分类挂载的相关技术研究

doi:10.3969／j.issn.1000-0801.2013.11.002

Abstract

Abstract:

In order to achieve the precise localization of internet customers' preference, the crucial point is to catch the classified mapping of URL system through digging deep into the data of DPI users' online behavior data. Based on summarizing the process of DPI data auto classification, the key technique was investigated which involved in Web information extraction, Chinese word segmentation, feature selection, text classification and so on. It may make solid technical foundations for the customer insight with DPI data.

Key words: deep packet inspection, Chinese word segmentation, feature selection, text classification

Lingyan Bian,Renlong He,Xiaohui Yao. Research on URL Classification with DPI Data Mining and Related Technology[J]. Telecommunications Science, 2013, 29(11): 6-11.

Figures/Tables 6

References 17

1	罗忆祖． DPI技术力助运营商精细化运营．邮电设计技术， 2009（3）
2	于静．基于页面主体提取的Web信息抽取技术研究．南京邮电大学硕士学位论文， 2013
3	冯书晓，徐新，杨春梅．国内外中文分词技术研究新进展．情报杂志， 2002（11）:29～30
4	郭瞳康．基于词典的中文分词技术研究．哈尔滨理工大学硕士学位论文， 2010
5	李原．中文文本分类中分词和特征选择方法研究．吉林大学硕士学位论文， 2011
6	薛为民，陆玉昌．文本挖掘技术研究．北京联合大学学报（自然科学版）， 2005,19（4）:59～63
7	宋江．文本分类的特征选择方法研究．南京航空航天大学硕士学位论文， 2010
8	王法波．文本分类的特征选择和分类方法研究．山东大学硕士学位论文， 2011
9	Liu H , Motoda H . Feature Extraction, Construction and Selection:A Data Mining Perspective. USA: Kluwer Academic, 1998
10	Jain A , Zongker D . Feature selection: evaluation, application and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997（2）: 153～158
11	Gorvan A . Principal Manifolds for Data Visualisation and Dimension Reduction. New York: Springer, 2007
12	Verleysen M , Lee J A . Rank-based quality assessment of nonlinear dimensionality reduction. Proceedings of the 16th European Symposium on Artificial Neural Networks, Bruges, Belgium, 2008: 49～54
13	Deerwester S . Indexing by latent semantic analysis. Journal of American Society for Information Science, 1990,41（6）: 391～407
14	陈燃燃．基于SVM算法的Web分类研究与实现．北京邮电大学硕士学位论文， 2009
15	Vapnik V N . The Nature of Statistical Learning Theory. New York: Springer, 1995
16	Joachims T . Text categorization with support vector machines:learning with many relevant features. Proceedings of the 10th European Conference on Machine Learning, Chemnitz, Germany, 1998: 137～142
17	Joachims T . Transductive inference for text classification using support vector machine s. Proceedings of the 16th International Conference on Machine Learning, Bled, Slovenia, 1999: 200～209

Metrics

Recommended 0

No Suggested Reading articles found!

编号	字段名	字段说明	关联分析内容
1	IMSI	用户的国际移动用户识别码	手机换卡分析
2	MDN	用户手机号码	移动用户手机号码识别
3	MEID	移动台设备标识	上网用户识别，防止盗用
4	NAI	网络接入标识	识别用户的上网账号
5	DestinationIP	用户访问的目标IPv4或IPv6地址	用户使用的应用、自动连接程序分析
6	DestinationPort	用户访问的目标端口号	用户使用的应用、自动连接程序分析
7	SourceIP	用户访问外部网络时使用的IPv4或IPv6地址	用户接入地址分析
8	SourcePort	用户访问外部网络时使用的端口号	用户接入地址分析
9	BSID	基站标识	用户进行用户位置分析与定位
10	Subnet	子网号，cdma20001xEv-Do系统的子网和扇区标识	用户进行更准确的用户位置分析与定位
11	ServiceOption	用户所接入的CDMA网络类型	用户上网的方式判断，如WAP、cdma2000 1x Ev-Do、Wi-Fi等
12	ProtocolID	协议类型	判断用户上网内容使用的通信协议
13	ServiceType	业务类型	判断用户所上网的内容，系统后续细化移动互联网判断的内容
14	StartTime	业务流开始时间	判断用户上网时间
15	EndTime	业务流结束时间	判断用户离开时间
16	DURATION	持续时间，单位ms	判断用户上网时长
17	Input-Octets	发给用户的业务字节数	判断用户下行数据量
18	Output-Octets	用户发出的业务字节数	判断用户上行数据量

协议	编号	字段名	字段说明	关联分析内容
HTTP	19	DestinationURL	用户访问的目标网站的URL	用户访问网址分析
	20	UserAgent	User Agent信息	识别终端，浏览器版本信息
	21	DomainName	外部网站的域名	用户访问网址和应用识别分析
	22	HOST	主机名	主机名
WAP	23	UserAgent	User Agent信息	识别终端，浏览器版本信息
	24	DestinationURL	用户访问的目标网站的URL	用户访问网址分析
	25	DomainName	外部网站的域名	用户访问网址和应用识别分析
	26	HOST	主机名	主机名
	27	WAPVERSION	WAP版本	ABP版本识别
	28	ACT_TYPE	WAP操作类型（5:Post,6:Get）	ABP操作类型识别

[1]	Jin ZHAO, Xiaojun YANG. Application on text classification of telecom user complaints based on GRW and FastText model [J]. Telecommunications Science, 2021, 37(6): 125-131.
[2]	Guoqing LIU, Xingqi WANG, Dan WEI, Jinglong FANG, Yanli SHAO. Feature selection method for software defect number prediction based on maximum information coefficient [J]. Telecommunications Science, 2021, 37(5): 133-147.
[3]	Liang ZHANG, Xiaoju DAI, Rong ZHENG, Tongze He. Research and implementation of text classification method for customer service orders based on multi-model fusion [J]. Telecommunications Science, 2021, 37(11): 86-96.
[4]	Feng LIN,Liujing XU,Xiaohua CHEN,Weiqiang QI,Ke CHEN,Tiantian ZHU. Method of Webshell detection based on multi-view feature fusion [J]. Telecommunications Science, 2020, 36(6): 125-132.
[5]	Jianwu ZHANG,Jiajia YANG,Zhendong WU. Method of fingerprint and voiceprint based decision level fusion recognition [J]. Telecommunications Science, 2018, 34(3): 32-40.
[6]	Shaomin WANG,Di YANG,Hua REN. Key technology research and model validation of text classification system based on deep learning [J]. Telecommunications Science, 2018, 34(12): 117-124.
[7]	Mingxin LIU,Jing CHEN,Qiyuan WANG. Research on text sentiment classification based on improved feature selection method [J]. Telecommunications Science, 2018, 34(10): 85-95.
[8]	Chunqin ZANG,Lichun XIE. Network intrusion detection method based on improved FCM and rule parameter optimization in cloud environment [J]. Telecommunications Science, 2018, 34(1): 72-79.
[9]	Xiaoyuan XU,Li HUANG. A feature selection method based on instance learning and cooperative subset search [J]. Telecommunications Science, 2017, 33(6): 105-113.
[10]	Xiaosong LIU,Yi’an MA. Research and implementation of mobile DPI association algorithm with big data technology [J]. Telecommunications Science, 2017, 33(12): 121-126.
[11]	Xiaohuai KOU,Hua CHENG. Design and implementation of spam filtering system based on topic model [J]. Telecommunications Science, 2017, 33(11): 73-82.
[12]	Yizhi DING,Hongwei LV,Yingxia NIU,Shaoping LI. Discussion on the deployment solution of comprehensive traffic management [J]. Telecommunications Science, 2016, 32(6): 195-200.
[13]	Kang CHEN,Huazheng FU,Chong CHEN,Yuyi ZHANG,Yu WEI. A real time approach to user interest classification using DPI [J]. Telecommunications Science, 2016, 32(12): 109-115.
[14]	Hao CHEN,Sihan QING. Android malware detection method based on combined algorithm [J]. Telecommunications Science, 2016, 32(10): 15-21.
[15]	Jian Wang,Zhizhong Zhang,Yunlong Luo. Design and Implementation of DPI-Based User's Bebavior Perception System in LTE Network [J]. Telecommunications Science, 2014, 30(7): 77-83.

Research on URL Classification with DPI Data Mining and Related Technology

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 17

Related Articles 15

Metrics

Recommended 0