智能运维中KPI异常检测的研究进展

doi:10.11959/j.issn.1000-0801.2021105

电信科学 ›› 2021, Vol. 37 ›› Issue (5): 42-51.doi: 10.11959/j.issn.1000-0801.2021105

• 专题：通信与AI融合 • 上一篇下一篇

智能运维中KPI异常检测的研究进展

王速¹, 卢华², 汪硕¹^,³, 蔡磊², 黄韬¹^,³

¹ 北京邮电大学网络与交换国家重点实验室, 北京 100876
² 广东省新一代通信与网络创新研究院，广东广州 510663
³ 网络通信与安全紫金山实验室，江苏南京 211111

修回日期:2021-05-10 出版日期:2021-05-20 发布日期:2021-05-01
作者简介:王速（1997− ），男，北京邮电大学硕士生，主要研究方向为软件定义网络、网络人工智能
卢华（1976− ），男，广东省新一代通信与网络创新研究院网络技术创新中心主任，主要研究方向为核心网、软件定义网络、P4可编程、虚拟化等
汪硕（1991− ），男，博士，北京邮电大学讲师，主要研究方向为软件定义网络、数据中心网络、确定性网络、网络人工智能等
蔡磊（1981− ），男，广东省新一代通信与网络创新研究院研究员，主要研究方向为可编程交换技术、网络流量控制技术等
黄韬（1980− ），男，博士，北京邮电大学教授，主要研究方向为路由与交换、软件定义网络、网络试验设施等
基金资助:
广东省重点领域研发计划基金资助项目(2018B010113001);国家重点研发计划基金资助项目(2018YFB1800500);国家自然科学基金资助项目(61902033);北京市自然科学基金资助项目(4204105)

Research progress of KPI anomaly detection in intelligent operation and maintenance

Su WANG¹, Hua LU², Shuo WANG¹^,³, Lei CAI², Tao HUANG¹^,³

¹ State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
² Guangdong Communications ＆ Networks Institute, Guangzhou 510663, China
³ Purple Mountain Laboratories, Nanjing 211111, China

Revised:2021-05-10 Online:2021-05-20 Published:2021-05-01
Supported by:
The Research and Development Program in Key Areas of Guangdong Province(2018B010113001);The National Key Research and Development Program of China(2018YFB1800500);The National Natural Science Foundation of China(61902033);Beijing Natural Science Foundation(4204105)

摘要/Abstract

摘要：

现有的网络监控和故障修复大多依赖规则系统或者人工处理，然而随着网络规模的不断增大和业务的多样化，这种方式难以满足要求。随着机器学习和深度学习等技术的快速发展，智能运维理论也取得了长足进步，利用人工智能技术提升网络运维智能化能力。KPI（key performance indicator）异常检测是智能运维的一项底层核心技术。针对KPI异常检测技术研究展开综述，对KPI数据和KPI异常进行了描述，并从单指标和多指标两个方面详细介绍了KPI异常检测技术的研究现状；分析了KPI检测的部署应用问题，讨论了未来的研究方向。

关键词: 智能运维, KPI, 异常检测

Abstract:

Existing network monitoring and fault repair mostly rely on rule systems or manual processing.However, the increase in network scale and the diversification of services make this approach difficult to deal with.With the rapid development of technology such as machine learning and deep learning, intelligent operation and maintenance theory has also made great progress, using artificial intelligence technology to enhance the intelligent ability of network operation and maintenance.KPI (key performance indicator) anomaly detection is an underlying core technology of intelligent operation and maintenance.A survey on the KPI anomaly detection technology was given.Firstly, the KPI data and KPI anomalies were described.Then the research progress of single-dimensional KPI and multi-dimensional KPI anomaly detection were introduced.Then, the deployment and application problems of KPI anomaly detection were analyzed.Finally, future research directions were discussed.

Key words: intelligent operation and maintenance, KPI, anomaly detection

中图分类号:

TP393

王速, 卢华, 汪硕, 蔡磊, 黄韬. 智能运维中KPI异常检测的研究进展[J]. 电信科学, 2021, 37(5): 42-51.

Su WANG, Hua LU, Shuo WANG, Lei CAI, Tao HUANG. Research progress of KPI anomaly detection in intelligent operation and maintenance[J]. Telecommunications Science, 2021, 37(5): 42-51.

图/表 4

参考文献 41

[1]	李青 . 基于大数据分析的云资源池告警信息关联方案[J]. 电信科学, 2020,36(10): 159-171.
	LI Q . Alarm information association scheme of cloud resource pool based on big data analysis[J]. Telecommunications Science, 2020,36(10): 159-171.
[2]	裴丹, 张圣林, 裴昶华 . 基于机器学习的智能运维[J]. 中国计算机学会通讯, 2017,13(12): 67-73.
	PEI D , ZHANG S L , PEI C H . Intelligent operation and main-tenance based on machine learning[J]. Communications of CCF, 2017,13(12): 67-73.
[3]	CHEN W X , XU H W , LI Z Y ,et al. Unsupervised anomaly detection for intricate KPIs via adversarial training of VAE[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. Piscataway: IEEE Press, 2019: 1891-1899.
[4]	CHANDOLA V , BANERJEE A , KUMAR V . Anomaly detection[J]. ACM Computing Surveys, 2009,41(3): 1-58.
[5]	Amazon Web Services. Amazon cloudwatch alarm[EB]. 2018.
[6]	SIFFER A , FOUQUE P A , TERMIER A ,et al. Anomaly detection in streams with extreme value theory[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2017: 1067-1075.
[7]	ZHANG Y , GE Z H , GREENBERG A ,et al. Network anomography[C]// Proceedings of the 5th ACM SIGCOMM conference on Internet measurement - IMC '05. New York: ACM Press, 2005: 30-30.
[8]	YAN H , FLAVEL A , GE Z H ,et al. Argus: End-to-end service anomaly detection and localization from an ISP's point of view[C]// Proceedings of 2012 IEEE INFOCOM. Piscataway:IEEE Press, 2012: 2756-2760.
[9]	TAYLOR S J , LETHAM B . Forecasting at scale[J]. The American Statistician, 2018,72(1): 37-45.
[10]	BREUNIG M M , KRIEGEL H P , NG R T ,et al. LOF: identifying density-based local outliers[C]// Proceedings of the 2000 ACM SIGMOD International Conference on Management of data - SIGMOD '00. New York: ACM Press, 2000: 93-104.
[11]	KRIEGEL H P , S HUBERT M , ZIMEK A . Angle-based outlier detection in high-dimensional data[C]// Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08. New York:ACM Press, 2008: 444-452.
[12]	LAPTEV N , AMIZADEH S , FLINT I . Generic and scalable framework for automated time-series anomaly detection[C]// Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2015: 1939-1947.
[13]	LIU D P , ZHAO Y J , XU H W ,et al. Opprentice: towards practical and automatic anomaly detection through machine learning[C]// Proceedings of the 2015 Internet Measurement Conference. New York: ACM Press, 2015: 211-224.
[14]	AMER M , GOLDSTEIN M , ABDENNADHER S . Enhancing one-class support vector machines for unsupervised anomaly detection[C]// Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description - ODD '13. New York: ACM Press, 2013: 8-15.
[15]	YANG X W , LATECKI L J , POKRAJAC D . Outlier detection with globally optimal exemplar-based GMM[C]// Proceedings of the 2009 SIAM International Conference on Data Mining. Philadelphia: Society for Industrial and Applied Mathematics, 2009: 145-154.
[16]	MUNZ G , LI S , CARLE G . Traffic anomaly detection using k-means clustering[C]// Proceedings of GI/ITG Workshop MMBnet.[S.l.:s.n.], 2007: 13-14.
[17]	AN J , CHO S . Variational autoencoder based anomaly detection using reconstruction probability[J]. Special Lecture on IE, 2015,2(1): 1-18.
[18]	GOODFELLOW I , POUGET-ABADIE J MIRZA M ,et al. Generative adversarial nets[C]// Advances in neural information processing systems.[S.l.:s.n.], 2014: 2672-2680.
[19]	OORD A , KALCHBRENNER N , KAVUKCUOGLU K . Pixel recurrent neural networks[J]. arXiv preprint arXiv:1601.06759, 2016.
[20]	XU H W , FENG Y , CHEN J ,et al. Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications[C]// Proceedings of the 2018 World Wide Web Conference on World Wide Web-WWW '18. New York:ACM Press, 2018: 187-196.
[21]	LI Z Y , CHEN W X , PEI D . Robust and unsupervised KPI anomaly detection based on conditional variational autoencoder[C]// Proceedings of 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC). Piscataway:IEEE Press, 2018: 1-9.
[22]	CHEN W X , XU H W , LI Z Y ,et al. Unsupervised anomaly detection for intricate KPIs via adversarial training of VAE[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.[S.l.:s.n.] Piscataway:IEEE Press, 2019: 1891-1899.
[23]	ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein gan[J]. arXiv preprint arXiv:1701.07875, 2017.
[24]	胡珉, 白雪, 徐伟 ,等. 多维时间序列异常检测算法综述[J]. 计算机应用, 2020,40(6): 1553-1564.
	HU M , BAI X , XU W ,et al. Review of anomaly detection algo-rithms for multidimensional time series[J]. Journal of Computer Applications, 2020,40(6): 1553-1564.
[25]	Arundo Analytics. Anomaly detection toolkit[EB]. 2019.
[26]	GOLDSTEIN M , DENGEL A . Histogram-based outlier score (hbos): a fast unsupervised anomaly detection algorithm[J]. KI-2012: Poster and Demo Track, 2012: 59-63.
[27]	陈兴蜀, 江天宇, 曾雪梅 ,等. 基于多维时间序列分析的网络异常检测[J]. 工程科学与技术, 2017,49(1): 144-150.
	CHEN X S , JIANG T Y , ZENG X M ,et al. Network anomaly detector based on multiple time series analysis[J]. Advanced Engineering Sciences, 2017,49(1): 144-150.
[28]	HU M , JI Z W , YAN K ,et al. Detecting anomalies in time series data via a meta-feature based approach[J]. IEEE Access, 2018(6): 27760-27776.
[29]	HUNDMAN K , CONSTANTINOU V , LAPORTE C ,et al. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining. New York: ACM Press, 2018: 387-395.
[30]	MALHOTRA P , RAMAKRISHNAN A , ANAND G ,et al. LSTM-based encoder-decoder for multi-sensor anomaly detection[J]. arXiv preprint arXiv:1607.00148, 2016.
[31]	PARK D , HOSHI Y , KEMP C C . A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018,3(3): 1544-1551.
[32]	SU Y , ZHAO Y J , NIU C H ,et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining. New York: ACM Press, 2019: 2828-2837.
[33]	ZHANG C X , SONG D J , CHEN Y C ,et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019(33): 1409-1416.
[34]	ZHAO N W , ZHU J , WANG Y ,et al. Automatic and generic periodicity adaptation for KPI anomaly detection[J]. IEEE Transactions on Network and Service Management, 2019,16(3): 1170-1183.
[35]	MA M H , ZHANG S L , PEI D ,et al. Robust and rapid adaption for concept drift in software system anomaly detection[C]// Proceedings of 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). Piscataway: IEEE Press, 2018: 13-24.
[36]	LI Z H , ZHAO Y J , LIU R ,et al. Robust and rapid clustering of KPIs for large-scale anomaly detection[C]// Proceedings of 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS). Piscataway: IEEE Press, 2018: 1-10.
[37]	BU J H , LIU Y , ZHANG S L ,et al. Rapid deployment of anomaly detection models for large number of emerging KPI streams[C]// Proceedings of 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC). Piscataway: IEEE Press, 2018: 1-8.
[38]	CHENG Z Q , YANG Y , WANG W ,et al. Time2Graph: revisiting time series modeling with dynamic shapelets[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 3617-3624.
[39]	SHEN Z Y , CUI P , LIU J S ,et al. Stable learning via differentiated variable decorrelation[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining. New York: ACM Press, 2020: 2185-2193.
[40]	HUANG Q B , WEI J L , CAI Y ,et al. Aligned dual channel graph convolutional network for visual question answering[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online. Stroudsburg: Association for Computational Linguistics, 2020: 7166-7176.
[41]	CHANG H , RONG Y , XU T Y ,et al. A restricted black-box adversarial framework towards attacking graph embedding models[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 3389-3396.

智能运维中KPI异常检测的研究进展

Research progress of KPI anomaly detection in intelligent operation and maintenance

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 41

相关文章 15

Metrics

推荐阅读 0

[1]	张海龙, 李博, 贾娜娟. 基于Dijkstra算法的电力光通信网络智能运维设备脱网故障区段定位方法[J]. 电信科学, 2023, 39(1): 92-99.
[2]	赵东明. 电信运营商知识图谱技术体系研究及应用实践[J]. 电信科学, 2022, 38(8): 151-162.
[3]	陈华华, 陈哲. 基于钉板分布稀疏变分自编码器的异常检测算法研究[J]. 电信科学, 2022, 38(12): 65-77.
[4]	程瑞营, 张攀, 肖雨, 乔宇杰, 张安奕. 基于时序数据的云网协同平台人工智能运维体系[J]. 电信科学, 2022, 38(11): 24-35.
[5]	韩赛, 张冬月, 王泽林, 王光全, 李奥, 方遒铿, 马红兵. 跨专业承载网络智能运维研究与应用[J]. 电信科学, 2022, 38(11): 113-122.
[6]	陈华华, 陈哲, 郭春生, 应娜, 叶学义, 章坚武. 基于混合高斯变分自编码网络的异常检测算法[J]. 电信科学, 2021, 37(4): 54-61.
[7]	陆钢,陈长怡,黄泽龙,黄泽源. 面向云网融合的智能云原生架构和关键技术研究[J]. 电信科学, 2020, 36(9): 67-74.
[8]	沈潇军,葛亚男,沈志豪,倪阳旦,吕明琪,翁正秋. 一种基于LSTM自动编码机的工业系统异常检测方法[J]. 电信科学, 2020, 36(7): 136-145.
[9]	吴亚萍,董红召,林盈盈,柳菁. 混沌改进鱼群算法及其在工业控制网络异常检测中的应用[J]. 电信科学, 2020, 36(3): 27-33.
[10]	薛龙,陆钢,周奇,张会炎,万亭君. 面向云原生的智能运维架构和关键技术[J]. 电信科学, 2020, 36(12): 105-112.
[11]	程卫华,何肖嵘. 基于大数据的网页浏览质差分析方法研究[J]. 电信科学, 2020, 36(11): 174-181.
[12]	李青. 基于大数据分析的云资源池告警信息关联方案[J]. 电信科学, 2020, 36(10): 159-171.
[13]	谭晓敏,方艾,梁冰,杨豪杰. 基于决策树的EPG体验异常原因定位[J]. 电信科学, 2019, 35(8): 158-164.
[14]	谭晓敏,方艾,金铎,李长江. IPTV用户体验异常的自动化检测[J]. 电信科学, 2019, 35(7): 159-164.
[15]	陈天骄,刘江,黄韬. 人工智能在网络编排系统中的应用[J]. 电信科学, 2019, 35(5): 9-16.