未知网络应用流量的自动提取方法

doi:10.3969/j.issn.1000-436x.2014.07.020

摘要/Abstract

摘要：

提取未知网络应用特征时需要获得其流量数据，但在网络工程中，采集的未知应用流量往往是几种应用流量的混合，如何将未知混合流量进行分离，按照应用进行归类是现有方法没有解决的问题。基于此提出一种基于载荷信息的流量聚类方法，该方法通过对报文载荷的部分字节编码，采用扩展的ROCK算法对未知混合流量进行分离，按照不同应用进行归类。实验结果表明，与基于会话行为特征（一种流量统计特征）的流量聚类方法相比，这种方法具有较高的精确度。

关键词: 流量分类, 会话行为特征, 载荷, ROCK算法

Abstract:

The features of unknown network applications can be extracted using its traffic data. However, the sample traffic in network engineering is usually a mixed traffic generated by several unknown applications. The separation of the mixed traffic by applications an unsolved problem presently. A clustering method for traffic classification was proposed based on payload information. The proposed method can firstly encode certain bytes of message payload, then separate and classify the unknown mixed traffic using an extended ROCK algorithm. The experiment results reveal that compared with the clustering method based on statistics character of traffic, the proposed method has higher accuracy.

Key words: traffic classification, behavioral features of session, payload, ROCK algorithm

王变琴,余顺争. 未知网络应用流量的自动提取方法[J]. 通信学报, 2014, 35(7): 164-171.

Bian-qin WANG,Shun-zheng YU. Automatic extraction for the traffic of unknown network applications[J]. Journal on Communications, 2014, 35(7): 164-171.

图/表 7

图1

表1

表2

表3

表4

表5

表6

参考文献 26

[1]	BERNAILLE L , TEIXEIRA R , AKODKENKUT I , et al. Traffic classification on the fly[J]. ACM SIGCOMM Computer Communica-tion Review, 2006,36(2):23-26.
[2]	CROTTI M , DUSI M , GRINFOLI F , et al. Traffic classification through simple statistical fingerprinting[J]. ACM SIGCOMM Computer Communication Review, 2007,37(1):1-16.
[3]	SEN S , SPATSCHECK O , WANG D M . Accurate, scalable in-network identification of P2P traffic using application signatures[A]. Proceed-ings of WWW2004[C]. New York, 2004. 512-521.
[4]	Application layer packet classifier for linux[EB/OL]. , http://17-filter.Sourceforge.net, 2012.
[5]	PARK BC , WON YJ , KIM MS , et al. Towards automated application signature generation for traffic identification[A]. Proceeding of the IEEE/IFIP Network Operations and Management Symposium[C]. Salvador da Bahia, 2008. 160-167.
[6]	WANG Y , XIANG Y , ZHOU W L , et al. Generating regular expression signatures for network traffic classification in trusted network man-agement[J]. Journal of Network and Computer Applications, 2011,17:1-9.
[7]	YE M J , XU K , WU J P , et al. Autosig-automatically generating sig-natures for applications[A]. Proceeding of the IEEE Ninth Interna-tional Conference on Computer and Information Technology[C]. Xia-men, China, 2009. 104-109.
[8]	赵咏，姚秋林，张志斌等. TPCAD：一种文本类多协议特征自动发现方法[J]. 通信学报， 2009,30(10A):28-35. ZHAO Y , YAO Q L , ZHANG Z B , et al. TPCAD: a text-oriented multi-protocol inference approach[J]. Journal on Communications, 2009,30(10A):28-35.
[9]	刘兴彬，杨建华，谢高岗等. 基于Apriori算法的流量识别特征自动提取方法[J]. 通信学报， 2008,29(12):51-59. LIU X B , YANG J H , XIE G G , et al. Automated mining of packet signatures for traffic identification at application layer with Apriori algorithm[J]. Journal on Communications, 2008,29(12):51-59.
[10]	龙文，马坤，辛阳等. 适用于协议特征提取的关联规则改进算法[J]. 电子科技大学学报， 2010,39(2):302-305. LONG W , MA K , XIN Y , et al. Improved association rules algorithm for protocol signatures extracting[J]. Journal of University of Elec-tronic Science and Technology of China, 2010,39(2):302-305.
[11]	王变琴，余顺争 . 一种自适应网络应用特征发现方法[J]. 通信学报， 2013,34(3):127-137. WANG B Q , YU S Z . Adaptive extraction method of network applica-tion signatures[J]. Journal on Communications, 2013,34(3):127-137.
[12]	ZHANG M W , LIU D P . Scalable and accurate application signature discovery[A]. Proceeding of the IEEE Pacific-Asia Workshop on Com-putational Intelligence and Industrial Application[C]. 2008. 482-487.
[13]	MA J , LEVCHENKO K , KREIBICH C , et al. Unexpected means of protocol inference[A]. Proceeding of the 6th ACM SIGCOMM Con-ference on Internet Measurement[C]. New York, NY: ACM, 2006:313-326.
[14]	李伟明，张爱芳，刘建财等. 网络协议的自动化模糊测试漏洞挖掘方法[J]. 计算机学报， 2011,34(2):242-254. LI W M , ZHANG A F , LIU J C , et al. Automatic network protocol fuzz testing and vulnerability discovering method[J]. Chinese Journal of Computers, 2011,34(2):242-254.
[15]	MCGREGOR A , HALL M , LORIER P , et al. Flow clustering using machine learning techniques[A]. Proceedings of PAM'04[C]. Antibes Juan-les-Pins, France, 2004. 205-214.
[16]	ZANDER S , NGUYEN T , ARMITAGE G . Automated traffic classifi-cation and application identification using machine learning[A]. Pro-ceeding of LCN'05[C]. Sydney, Australia, 2005.
[17]	ERMAN J , ARLITT M ,and MAHANTI A . Traffic classification clustering algorithms[A]. Proceedings of SIGMETRICS'06 (Mine-Net)[C]. Pisa, Italy, 2006. 281-286.
[18]	董仕，王岗 . 基于UDP流量的P2P流媒体流量识别算法研究[J]. 通信学报， 2012,33(12):25-34. DONG S , WANG G . Research on P2P streaming media identification based on UDP[J]. Journal on Communications, 2012,33(12):25-34.
[19]	MOORE A W , ZUEV D . Discriminators for Use in Flow-based Classi-fication[R]. Intel Research, Cambridge, 2005.
[20]	BERNAILLE L , TEIXEIRA R , AKODKENOU I , et al. Traffic classi-fication on the fly[J]. ACM SIGCOMM Computer Communication Review, 2006,36(2):23-26.
[21]	BERNAILLE L , TEIXEIRA R , SALAMTIAN K . Early application identification[A]. Proceedings of CoNEXT'06[C], Lisboa, Portugal, 2006.
[22]	JAIN A K , DUBES R C . Algorithms for clustering data[M]. Pren-tice-Hall, Inc, 1988.
[23]	DUMPSTER A P , PAIRD N M , RUBIN D B . Maximum likelihood from incomplete data via the EM algorithm[J]. Journal of the Royal Statistical Society, 1977,39(1):1-38.
[24]	ESTER M , KRIEGEL H P , SANDER J , et al. A density-based algo-rithm for discovering clusters in large spatial database with noise[A]. Proceedings of the International Conference on Knowledge Discovery in Databases and Data Mining[C]. Portland, Oregon, 1996. 226-231.
[25]	GUHA S , RASTOGI R , SHIM K . ROCK: a robust clustering algo-rithm for categorical attributes[J]. Information System, 2000,25(5):345-366.
[26]	WEKAattributes[EB/OL]. . http://www.cs.waikato.ac.nz/~ml/weka/index.html.

应用	Session	Packet	Size/(Mbit·s^-1)
HTTP	880	57 568	77.2
FTP	920	20 616	102.24
SMTP	652	32 884	60.32
POP3	512	6 000 072	72.8
SSL	788	24 018	44.70
MSN	556	52 460	18.52
BT	1184	297 594	44.7

应用协议		FNR(方案1/方案2)			FPR(方案1/方案2)
应用协议	k-means	EM	DBSCAN	k-means	EM	DBSCAN
SSL	1.82/1.82	0.00/0.00	0.00/0.00	9.85/12.12	7.88/11.94	6.73/7.73
FTP	0.00/6.06	0.00/0.00	0.00/0.00	6.61/7.74	0.00/5.79	0.00/0.00
HTTP	30.0/30.3	30.0/30.3	17.02/17.02	0.00/0.00	0.00/0.00	0.00/0.00
注：参数设置：k-means (k=3)、EM (k=3)、DBSCAN(Eps=0.23,MinPts=6),DBSCAN会自动丢弃一些自认为是噪声的样本，在计算漏报率与误报率时，实际统计的是参与聚类的样本个数，而非全部样本个数。

应用协议		FNR			FPR
应用协议	k-means	EM	DBSCAN	k-means	EM	DBSCAN
SSL	1.82	1.82	0.00	3.48	0.00	0.00
FTP	0.00	98.00	0.00	4.21	0.00	0.00
HTTP	30.30	0.00	0.00	0.53	6.23	0.00
MSN	7.25	8.70	0.00	5.35	32.62	0.00
注：参数设置：k-means(k=4)、EM(k=4)、DBSCAN(Eps=0.048,MinPts=6)

应用协议		FNR			FPR
应用协议	k-means	EM	DBSCAN	k-means	EM	DBSCAN
SSL	1.82	1.82	0.00	3.08	0.00	0.00
FTP	0.00	98.00	0.00	13.59	0.00	3.91
HTTP	30.30	0.00	0.00	0.00	3.80	0.00
BT	31.75	1.59	4.93	5.35	32.62	0.00
注：参数设置：k-means(k=4)、EM(k=4)、DBSCAN(Eps=0.085,MinPts=6)

应用协议		FNR			FPR
应用协议	k-means	EM	DBSCAN	k-means	EM	DBSCAN
SSL	1.82	1.82	0.00	4.5	0.00	0.00
FTP	99.00	98.00	98.00	0.00	0.00	0.00
HTTP	21.21	0.00	0.00	0.53	7.41	0.00
POP3	1.47	5.88	0.00	38.50	30.48	42.66
注：参数设置：k-means(k=4)、EM(k=4)、DBSCAN(Eps=0.07,MinPts=6)