基于深度学习的域名查询行为向量空间嵌入

doi:10.11959/j.issn.1000-436x.2016064

摘要/Abstract

摘要：

提出一种新的分析 DNS 查询行为的方法，用深度学习机制将被查询域名和请求查询的主机分别映射到向量空间，域名或主机的关联分析转化成向量的运算。通过对2组真实的校园网DNS 日志数据集的处理，发现该方法很好地保持了关联特性，使用降维处理以及聚类分析，不仅可以让人直观地发现隐含的关联关系，还有助于发现网络中的异常问题如botnet等。

关键词: DNS, 深度学习, 上下文, 降维, 行为分析, 层次聚类

Abstract:

A novel approach to analyze DNS query behaviors was introduced.This approach embeds queried domains or querying hosts to vector space by deep learning mechan then the relationship between querying of domains or hosts was mapped to vector space operations.By processing two real campus network DNS log datasets,it is found that this method maintains relationships very well.After doing mension reduction and clustering analysis,researchers can not only easily explore hidden relationships intuitively,but also discover abnormal network events like botnet.

Key words: DNS, deep learning, context, dimension reduction, behavior analysis, hierarchical clustering

周昌令,栾兴龙,肖建国. 基于深度学习的域名查询行为向量空间嵌入[J]. 通信学报, 2016, 37(3): 165-174.

Chang-ling ZHOU,Xing-long LUAN,Jian-guo XIAO. Vector space embedding of DNS query behaviors by deep learning[J]. Journal on Communications, 2016, 37(3): 165-174.

图/表 10

图1

图2

表1

表2

图3

图4

图5

图6

图7

图8

参考文献 31

[1]	MOGHADDAM S , HELMY A . Spatio-temporal modeling of wireless users Internet access patterns using self-organizing maps[C]// 2011 Proceedings IEEE INFOCOM. c2011:496-500.
[2]	CAGLAYAN A , TOOTHAKER M , DRAPAEAU D , et al. Behavioral analysis of fast flux service networks[C]// 2010 43rd Hawaii Interna-tional Conference on System Sciences. c2009:1-9.
[3]	BILGE L , KIRDA E , KRUEGEL C , et al. EXPOSURE:finding mali-cious domains using passive DNS analysis[C]// NDSS. c2011:1-17.
[4]	ANTONAKAKIS M , PERDISCI R . From throw-away traffic to bots:detecting the rise of DGA-based malware[C]// The 21st USENIX Se-curity Symposium. c2012:24.
[5]	CHOI H , LEE H , LEE H , et al. Botnet detection by monitoring group activities in DNS traffic[C]// 7th IEEE International Confe-rence on Computer and Information Technology(CIT 2007). c2007:715-720.
[6]	CHEN Y , ANTONAKAKIS M . DNS noise:measuring the pervasive-ness of disposable domains in modern DNS traffic[C]// Dependable Systems and Networks (DSN),44th Annual IEEE/IFIP International Conference on. c2014:598-609.
[7]	CALLAHAN T , ALLMAN M , RABINOVICH M . On modern DNS behavior and properties[J]. ACM SIGCOMM Computer Commu ica-tion Review, 2013,43(3): 7.
[8]	MIKOLOV T , CHEN K , CORRADO G , et al. Efficient estimation of word representations in vector space[J]. arXiv Preprint arXiv.1301.3781.20B.
[9]	WIKIPEDIA. Embedding[EB/OL]. , 2015.
[10]	HINTON G E . Learning distributed representations of concepts[C]// The Eighth Annual Conference of the Cognitive Science Society. c1986:1-12.
[11]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553): 436-444.
[12]	REHUREK R . Word2vec in python,part two:optimizing.[EB/OL]. , 2015.
[13]	MIKOLOV T , SUTSKEVER I , CHEN K , et al. Distributed represen-tations of words and phrases and their compositionality[C]// Advances in Neural Information Processing Systems. c2013:3111-3119.
[14]	MAATEN L V D , HINTON G . Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008,9:2579-2605.
[15]	JAIN A , MURTY M , FLYNN P . Data clustering:a review[J]. ACM Computing Surveys(CSUR), 1999,31(3): 264-323.
[16]	WIKIPEDIA. Complete-linkage clustering - wikipedia,the free en-cyclopedia[EB/OL]. , 2015.
[17]	BRODER A , MITZENMACHER M . Network applications of bloom filters:a survey[J]. Internet Mathematics, 2004,1(4): 485-509.
[18]	FJELLSKAL E B . Passive DNS tool.[EB/OL]. , 2015.
[19]	马云龙，姜彩萍，张千里，等. 基于IPFIX 的DNS异常行为检测方法[J]. 通信学报， 2014,35(z1): 5-9. MA Y L , JIANG C P , ZHANG Q L , et al. DNS abnormal behavior detec-tion based on IPFIX[J]. Journal on Communications, 2014,35(z1): 5-9.
[20]	BOSTOCK M . Data driven documents.[EB/OL]. .
[21]	GAO H , YEGNESWARAN V , CHEN Y , et al. An empirical reexami-nation of global DNS behavior[J]. ACM SIGCOMM Computer Com-munication Review, 2013,43(4): 267-278.
[22]	CISCO. Cisco IOS NetFlow[EB/OL]. .
[23]	WIKIPEDIA. Entropy (information theory)-wikipedia,the free encyc-lopedia[EB/OL]. . 2015.
[24]	HERRMANN D , BANSE C , FEDERRATH H . Behaviorbased track-ing:exploiting characteristic patterns in DNS traffic[J]. Computers＆Security, 2013,39:17-33.
[25]	袁春阳，李青山，王永建 . 基于行为与域名查询关联的僵尸网络聚类联动监测[J]. 计算机应用研究， 2012,29(3): 1084-1087. YUAN C Y , LI Q S , WANG Y J . Linkage monitoring of clus for botnet based on relevance of behavior and domain inqui y[J]. Applica-tion Research of Computers, 2012,29(3): 1084-1087.
[26]	KRISHNAN S , TAYLOR T , MONROSE F , et al. Crossing the thre-shold:detecting network malfeasance via sequential hypothesis test-ing[C]// 2013 43rd Annual IEEE/IFIP International Conference on De-pendable Systems and Networks(DSN). c2013:1-12.
[27]	ZOU W Y , SOCHER R , CER D , et al. Bilingual word embeddings for phrase-based machine translation[C]// 2013 Conference on Empirical Methods in Natural Language Processing(EMNLP 2013). c2013:1393-1398.
[28]	LEVY O , GOLDBERG Y . Linguistic regularities in sparse and explicit word representations[C]// Proceedings of the 18th Conference on Computational Natural Language Learning(CoNLL 2014), c2014.
[29]	WIKIPEDIA. Pointwise mutual information — Wikipedia,the free encyclopedia[EB/OL]. .
[30]	PEROZZI B , SKIENA S . DeepWalk:online learning of social Re-presentations[C]// The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. c2014:701-710.
[31]	TANG J , QU M , WANG M , et al. LINE:Largescale Information Network Embedding[J]. arXiv preprint arXiv:1503.03578, 2015.

DNS日志产生速率	＞4 000条/秒
DNS日志峰值速率	＞20 000条/秒
正常查询日志总量	约350M条/天
失败查询(NxDomain等)	约2M条/天
DNS原始查询流量	200~400 GB/天
占用空间（压缩率约20%）	2~5 GB/天
被查询的不同域名数量	约3×10⁶/天
请求查询的不同主机数量	约10⁵/天
采集时间	20150301~20150319

DNS日志产生速率	＞1 000条/秒
DNS日志峰值速率	＞7 000条/秒
正常查询日志总量	约100M 条/天
被查询的不同域名数量	约7×10⁵/天
请求查询的不同主机数量	约2×10⁵/天
采集时间	20150315~20150319