基于密度聚类的网络性能故障大数据分析方法

doi:10.11959/j.issn.1000-0801.2020270

Abstract

Abstract:

Facing frequent network security incidents,how to quickly find abnormal data in massive monitoring database and carry out network failure analysis becomes a research difficulty.A density-based network performance failure big data analysis algorithm was proposed,which extracted key performance characteristic indicators through entropy weight analysis,implemented data shaping through data cleaning and standardization,and extracted abnormal performance data on the basis of DBSCAN clustering algorithm.Relying on the real-time massive backbone network link performance data of multiple domestic operators to validated this algorithm,the results shows that compared with the manually manner,the recognition accuracy of the algorithm proposed to the network performance abnormal data is more than 90%,which can well fit for the analysis of real-time Internet network operation failure.

Key words: network performance, machine learning, density clustering, measurement analysis

CLC Number:

TP393

Xiang LI,Yuan LI,Zifei ZHANG,Zhe YANG. A density clustering-based network performance failure big data analysis algorithm[J]. Telecommunications Science, 2020, 36(9): 51-58.

Figures/Tables 9

References 14

[1]	李玥 . 机器学习的分类、聚类研究[J]. 电脑知识与技术, 2020,16(4): 161-162.
	LI Y . Research on classification and clustering of machine learning[J]. Computer Knowledge and Technology, 2020,16(4): 161-162.
[2]	蔡志荣 . 基于熵权的模糊综合评价法在学习质量评价中的应用[J]. 计算机时代, 2018.(12): 75-77.
	CAI Z R . Application of entropy weight based fuzzy comprehensive evaluation method in learning quality evaluation[J]. Computer Era, 2018.(12): 75-77.
[3]	刘竞妍, 张可, 王桂华 . 综合评价中数据标准化方法比较研究[J]. 数字技术与应用, 2018,36(6): 84-85.
	LIU J Y , ZHANG K , WANG G H . Comparative study on data standardization methods in comprehensive evaluation[J]. Digital Technology ＆ Application, 2018,36(6): 84-85.
[4]	展金梅, 陈君涛 . 聚类集成算法中度量方法[J]. 电子技术与软件工程, 2020(3): 170-171.
	CHEN J M , CHEN J T . Measurement method in clustering integration algorithm[J]. Electronic Technology ＆ Software Engineering, 2020(3): 170-171.
[5]	贺玲, 吴玲达, 蔡益朝 . 数据挖掘中的聚类算法综述[J]. 计算机应用研究, 2017,(1): 10-13.
	HE L , WU L D , CAI Y C . Survey of clustering algorithms in data mining[J]. Application Research of Computers, 2017(1): 10-13.
[6]	周爱武, 于亚飞 . K-means聚类算法的研究[J]. 计算机技术与发展, 2011,21(2): 62-65.
	ZHOU A W , YU Y F . The research about clustering algorithm of K-means[J]. Computer Technology and Development, 2011,21(2): 62-65.
[7]	闫玮 . 基于多种层次聚类的算法研究[D]. 西安:西安电子科技大学, 2019.
	YAN W . Algorithms research based on multiple hierarchical clustering[D]. Xi’an:Xidian University, 2019.
[8]	ESTER M , KRIEGEL H P , XU X . A density-based algorithm for discovering clusters adensity-based algorithm for discovering clusters in large spatial databases with noise[C]// Proceedings of International Conference on Knowledge Discovery ＆ Data Mining. New York:ACM Press, 1996.
[9]	谢娟英, 丁丽娟 . 完全自适应的谱聚类算法[J]. 电子学报, 2019,47(5): 1000-1008.
	XIE J Y , DING L J . The true self-adaptive spectral clustering algorithms[J]. Acta Electronica Sinica, 2019,47(5): 1000-1008.
[10]	孙鹏, 韩承德, 曾涛 . S-DBSCAN:一种基于DBSCAN发现高密度簇的算法[J]. 高技术通讯, 2012,22(6): 589-595.
	SUN P , HAN C D , ZENG T . S-DBSCAN:an algorithm for finding high density clusters based on DBSCAN[J]. Chinese High Technology Letters, 2012,22(6): 589-595.
[11]	SHAH G H , . An improved DBSCAN,a density based clustering algorithm with parameter selection for high dimensional data sets[C]// Proceedings of Nirma University International Conference on Engineering. Piscataway:IEEE Press, 2013.
[12]	冯振华 . 基于 DBSCAN 聚类算法的研究与应用[D]. 无锡:江南大学, 2016.
	FENG Z H . Research and application of clustering algorithm based on DBSCAN[D]. Wuxi:Jiangnan University, 2016.
[13]	SUNITA J , PATAG K . Algorithm to determine ε-distance parameter in density based clustering[J]. Expert Systems With Applications, 2014,(6): 2939-2946.
[14]	冯万兴, 朱晔, 郭钧天 ,等. 基于改进的 DBSCAN 方法和多项式拟合的雷电短时预测[J]. 计算机工程与科学, 2014,36(10): 2028-2033.
	FENG W X , ZHU Y , GUO J T ,et al. Lightning forecast based on the improved DBSCAN and polynomial fitting[J]. Computer Engineering and Science, 2014,36(10): 2028-2033.

Metrics

Recommended 0

No Suggested Reading articles found!

链路方向	时延指标权重	分组丢失率指标权重	跳数指标权重
上海电信-上海电信	0.99	0.005	0.005
上海电信-四川联通	0.011 3	0.987 9	0.000 8
宁夏电信-安徽鹏博士	0.191 5	0.726 4	0.082 1

链路方向		eps值
链路方向	默认值0.8	曲线多项式拟合并求差值	指数拟合并求导	计算相邻点之间斜率
上海电信-上海电信	0.8	0.127 2	0	0.127 2
上海电信-四川联通	0.8	3.541 5	3.325 3	3.340 7
宁夏电信-安徽鹏博士	0.8	0.864 2	0.746 0	0.842 0

对比项	湖南电信-安徽移动	江苏移动-浙江电信	辽宁移动-浙江电信	安徽电信-浙江电信	吉林联通-山西电信
机器识别异常记录数	93/个	5/个	25/个	10/个	49/个
人工判断异常记录数	102/个	5/个	28/个	8/个	45/个
识别准确率	91.2%	100%	89.2%	80%	91.8%

对比项	湖南电信-安徽移动	江苏移动-浙江电信	辽宁移动-浙江电信	安徽电信-浙江电信	吉林联通-山西电信
DBSCAN算法	91.2%	100%	89.2%	80%	91.8%
K-means++算法	88.2%	70%	75%	47.5%	93.75%
AGNES算法	89.2%	70%	92.8%	42.1%	91.8%

[1]	Zehua GUO, Haowen ZHU, Tongwen XU. Network modal innovation for distributed machine learning [J]. Telecommunications Science, 2023, 39(6): 44-51.
[2]	Yaqiong LIU, Zhe LYU, Yafei ZHAO, Guochu SHOU. A survey on AI techniques applied in the satellite communication/satellite Internet field [J]. Telecommunications Science, 2023, 39(2): 10-24.
[3]	Jianwu ZHANG, Yanjun AN, Huangyan DENG. A survey on DNS attack detection and security protection [J]. Telecommunications Science, 2022, 38(9): 1-17.
[4]	Yijiang LI, Huibiao YE, Renhua XIE, Jiali LOU, Danna ZHUANG, Chuanhuang LI. Intelligent prediction method of network performance based on graph neural network [J]. Telecommunications Science, 2022, 38(3): 143-157.
[5]	Zhichao ZHOU, Yi FENG, Xiaohan XIA, Yuyao FENG, Chao CAI, Jiahui QIU, Lihui YANG, Yunxiao WU. Outdoor location scheme with fingerprinting based on machine learning of mobile cellular network [J]. Telecommunications Science, 2021, 37(8): 85-95.
[6]	Ziyu XIAO. A review of key technologies for the future of the 2030 network——Beyond 5G [J]. Telecommunications Science, 2020, 36(9): 114-121.
[7]	Yaxing QIU,Xidong WANG,Sen BIAN,Lei YUE. Load balancing based on clustering analysis and deep learning for multi-frequency and multi-mode network [J]. Telecommunications Science, 2020, 36(7): 156-162.
[8]	Ye OUYANG,Aidong YANG,Fanyu MENG. A game theory-assisted machine learning methodology for subscriber churn behaviors detection [J]. Telecommunications Science, 2020, 36(6): 79-89.
[9]	Feng LIN,Liujing XU,Xiaohua CHEN,Weiqiang QI,Ke CHEN,Tiantian ZHU. Method of Webshell detection based on multi-view feature fusion [J]. Telecommunications Science, 2020, 36(6): 125-132.
[10]	Jiantao SHENG,Maofei CHEN,Dongxin LIU,Laifu WANG,Guoshui SHI,Huamin JIN. Detection of malicious domain name based on a classifier combination [J]. Telecommunications Science, 2020, 36(5): 47-55.
[11]	Tao PANG,Haihua QIU,Biying PAN. Research on artificial intelligence key technologies of mobile terminal [J]. Telecommunications Science, 2020, 36(5): 145-151.
[12]	Qizhu ZHONG. Research and application of VoLTE video call quality based on machine learning [J]. Telecommunications Science, 2020, 36(3): 156-165.
[13]	Zishan LIU,Qiang CHENG,Bo LV. A survey on key technologies of privacy protection for machine learning [J]. Telecommunications Science, 2020, 36(11): 18-27.
[14]	Xiaolian CHEN,Yi QIN,Jie ZHANG,Yajie LI,Haokun SONG,Huibin ZHANG. Optical fiber eavesdropping detection method based on machine learning [J]. Telecommunications Science, 2020, 36(11): 61-67.
[15]	Fei GUI,Yang CHENG,Dan LI,Sihong HONG. Internet intelligent routing architecture and algorithm [J]. Telecommunications Science, 2020, 36(10): 12-20.

A density clustering-based network performance failure big data analysis algorithm

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 14

Related Articles 15

Metrics

Recommended 0