智能运维中KPI异常检测的研究进展

doi:10.11959/j.issn.1000-0801.2021105

Abstract

Abstract:

Existing network monitoring and fault repair mostly rely on rule systems or manual processing.However, the increase in network scale and the diversification of services make this approach difficult to deal with.With the rapid development of technology such as machine learning and deep learning, intelligent operation and maintenance theory has also made great progress, using artificial intelligence technology to enhance the intelligent ability of network operation and maintenance.KPI (key performance indicator) anomaly detection is an underlying core technology of intelligent operation and maintenance.A survey on the KPI anomaly detection technology was given.Firstly, the KPI data and KPI anomalies were described.Then the research progress of single-dimensional KPI and multi-dimensional KPI anomaly detection were introduced.Then, the deployment and application problems of KPI anomaly detection were analyzed.Finally, future research directions were discussed.

Key words: intelligent operation and maintenance, KPI, anomaly detection

CLC Number:

TP393

Su WANG, Hua LU, Shuo WANG, Lei CAI, Tao HUANG. Research progress of KPI anomaly detection in intelligent operation and maintenance[J]. Telecommunications Science, 2021, 37(5): 42-51.

Figures/Tables 4

References 41

[1]	李青 . 基于大数据分析的云资源池告警信息关联方案[J]. 电信科学, 2020,36(10): 159-171.
	LI Q . Alarm information association scheme of cloud resource pool based on big data analysis[J]. Telecommunications Science, 2020,36(10): 159-171.
[2]	裴丹, 张圣林, 裴昶华 . 基于机器学习的智能运维[J]. 中国计算机学会通讯, 2017,13(12): 67-73.
	PEI D , ZHANG S L , PEI C H . Intelligent operation and main-tenance based on machine learning[J]. Communications of CCF, 2017,13(12): 67-73.
[3]	CHEN W X , XU H W , LI Z Y ,et al. Unsupervised anomaly detection for intricate KPIs via adversarial training of VAE[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. Piscataway: IEEE Press, 2019: 1891-1899.
[4]	CHANDOLA V , BANERJEE A , KUMAR V . Anomaly detection[J]. ACM Computing Surveys, 2009,41(3): 1-58.
[5]	Amazon Web Services. Amazon cloudwatch alarm[EB]. 2018.
[6]	SIFFER A , FOUQUE P A , TERMIER A ,et al. Anomaly detection in streams with extreme value theory[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2017: 1067-1075.
[7]	ZHANG Y , GE Z H , GREENBERG A ,et al. Network anomography[C]// Proceedings of the 5th ACM SIGCOMM conference on Internet measurement - IMC '05. New York: ACM Press, 2005: 30-30.
[8]	YAN H , FLAVEL A , GE Z H ,et al. Argus: End-to-end service anomaly detection and localization from an ISP's point of view[C]// Proceedings of 2012 IEEE INFOCOM. Piscataway:IEEE Press, 2012: 2756-2760.
[9]	TAYLOR S J , LETHAM B . Forecasting at scale[J]. The American Statistician, 2018,72(1): 37-45.
[10]	BREUNIG M M , KRIEGEL H P , NG R T ,et al. LOF: identifying density-based local outliers[C]// Proceedings of the 2000 ACM SIGMOD International Conference on Management of data - SIGMOD '00. New York: ACM Press, 2000: 93-104.
[11]	KRIEGEL H P , S HUBERT M , ZIMEK A . Angle-based outlier detection in high-dimensional data[C]// Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08. New York:ACM Press, 2008: 444-452.
[12]	LAPTEV N , AMIZADEH S , FLINT I . Generic and scalable framework for automated time-series anomaly detection[C]// Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2015: 1939-1947.
[13]	LIU D P , ZHAO Y J , XU H W ,et al. Opprentice: towards practical and automatic anomaly detection through machine learning[C]// Proceedings of the 2015 Internet Measurement Conference. New York: ACM Press, 2015: 211-224.
[14]	AMER M , GOLDSTEIN M , ABDENNADHER S . Enhancing one-class support vector machines for unsupervised anomaly detection[C]// Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description - ODD '13. New York: ACM Press, 2013: 8-15.
[15]	YANG X W , LATECKI L J , POKRAJAC D . Outlier detection with globally optimal exemplar-based GMM[C]// Proceedings of the 2009 SIAM International Conference on Data Mining. Philadelphia: Society for Industrial and Applied Mathematics, 2009: 145-154.
[16]	MUNZ G , LI S , CARLE G . Traffic anomaly detection using k-means clustering[C]// Proceedings of GI/ITG Workshop MMBnet.[S.l.:s.n.], 2007: 13-14.
[17]	AN J , CHO S . Variational autoencoder based anomaly detection using reconstruction probability[J]. Special Lecture on IE, 2015,2(1): 1-18.
[18]	GOODFELLOW I , POUGET-ABADIE J MIRZA M ,et al. Generative adversarial nets[C]// Advances in neural information processing systems.[S.l.:s.n.], 2014: 2672-2680.
[19]	OORD A , KALCHBRENNER N , KAVUKCUOGLU K . Pixel recurrent neural networks[J]. arXiv preprint arXiv:1601.06759, 2016.
[20]	XU H W , FENG Y , CHEN J ,et al. Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications[C]// Proceedings of the 2018 World Wide Web Conference on World Wide Web-WWW '18. New York:ACM Press, 2018: 187-196.
[21]	LI Z Y , CHEN W X , PEI D . Robust and unsupervised KPI anomaly detection based on conditional variational autoencoder[C]// Proceedings of 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC). Piscataway:IEEE Press, 2018: 1-9.
[22]	CHEN W X , XU H W , LI Z Y ,et al. Unsupervised anomaly detection for intricate KPIs via adversarial training of VAE[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.[S.l.:s.n.] Piscataway:IEEE Press, 2019: 1891-1899.
[23]	ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein gan[J]. arXiv preprint arXiv:1701.07875, 2017.
[24]	胡珉, 白雪, 徐伟 ,等. 多维时间序列异常检测算法综述[J]. 计算机应用, 2020,40(6): 1553-1564.
	HU M , BAI X , XU W ,et al. Review of anomaly detection algo-rithms for multidimensional time series[J]. Journal of Computer Applications, 2020,40(6): 1553-1564.
[25]	Arundo Analytics. Anomaly detection toolkit[EB]. 2019.
[26]	GOLDSTEIN M , DENGEL A . Histogram-based outlier score (hbos): a fast unsupervised anomaly detection algorithm[J]. KI-2012: Poster and Demo Track, 2012: 59-63.
[27]	陈兴蜀, 江天宇, 曾雪梅 ,等. 基于多维时间序列分析的网络异常检测[J]. 工程科学与技术, 2017,49(1): 144-150.
	CHEN X S , JIANG T Y , ZENG X M ,et al. Network anomaly detector based on multiple time series analysis[J]. Advanced Engineering Sciences, 2017,49(1): 144-150.
[28]	HU M , JI Z W , YAN K ,et al. Detecting anomalies in time series data via a meta-feature based approach[J]. IEEE Access, 2018(6): 27760-27776.
[29]	HUNDMAN K , CONSTANTINOU V , LAPORTE C ,et al. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining. New York: ACM Press, 2018: 387-395.
[30]	MALHOTRA P , RAMAKRISHNAN A , ANAND G ,et al. LSTM-based encoder-decoder for multi-sensor anomaly detection[J]. arXiv preprint arXiv:1607.00148, 2016.
[31]	PARK D , HOSHI Y , KEMP C C . A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018,3(3): 1544-1551.
[32]	SU Y , ZHAO Y J , NIU C H ,et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining. New York: ACM Press, 2019: 2828-2837.
[33]	ZHANG C X , SONG D J , CHEN Y C ,et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019(33): 1409-1416.
[34]	ZHAO N W , ZHU J , WANG Y ,et al. Automatic and generic periodicity adaptation for KPI anomaly detection[J]. IEEE Transactions on Network and Service Management, 2019,16(3): 1170-1183.
[35]	MA M H , ZHANG S L , PEI D ,et al. Robust and rapid adaption for concept drift in software system anomaly detection[C]// Proceedings of 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). Piscataway: IEEE Press, 2018: 13-24.
[36]	LI Z H , ZHAO Y J , LIU R ,et al. Robust and rapid clustering of KPIs for large-scale anomaly detection[C]// Proceedings of 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS). Piscataway: IEEE Press, 2018: 1-10.
[37]	BU J H , LIU Y , ZHANG S L ,et al. Rapid deployment of anomaly detection models for large number of emerging KPI streams[C]// Proceedings of 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC). Piscataway: IEEE Press, 2018: 1-8.
[38]	CHENG Z Q , YANG Y , WANG W ,et al. Time2Graph: revisiting time series modeling with dynamic shapelets[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 3617-3624.
[39]	SHEN Z Y , CUI P , LIU J S ,et al. Stable learning via differentiated variable decorrelation[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining. New York: ACM Press, 2020: 2185-2193.
[40]	HUANG Q B , WEI J L , CAI Y ,et al. Aligned dual channel graph convolutional network for visual question answering[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online. Stroudsburg: Association for Computational Linguistics, 2020: 7166-7176.
[41]	CHANG H , RONG Y , XU T Y ,et al. A restricted black-box adversarial framework towards attacking graph embedding models[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 3389-3396.

Metrics

Recommended 0

No Suggested Reading articles found!

Research progress of KPI anomaly detection in intelligent operation and maintenance

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 41

Related Articles 15

Metrics

Recommended 0

[1]	Hailong ZHANG, Bo LI, Najuan JIA. A method for locating fault sections of power optical communication network intelligent operation and maintenance equipment off-grid based on Dijkstra algorithm [J]. Telecommunications Science, 2023, 39(1): 92-99.
[2]	Dongming ZHAO. Research and application practice of knowledge graph technology system for telecom-operators [J]. Telecommunications Science, 2022, 38(8): 151-162.
[3]	Huahua CHEN, Zhe CHEN. Research on anomaly detection algorithm based on sparse variational autoencoder using spike and slab prior [J]. Telecommunications Science, 2022, 38(12): 65-77.
[4]	Huahua CHEN, Zhe CHEN, Chunsheng GUO, Na YING, Xueyi YE, Jianwu ZHANG. Anomaly detection algorithm based on Gaussian mixture variational auto encoder network [J]. Telecommunications Science, 2021, 37(4): 54-61.
[5]	Yusun FU. Evaluation criteria and solution of mobile communication network [J]. Telecommunications Science, 2020, 36(11): 28-38.
[6]	Weihua CHENG,Xiaorong HE. Quality analysis method of Web browsing based on big data [J]. Telecommunications Science, 2020, 36(11): 174-181.
[7]	Qing LI. Alarm information association scheme of cloud resource pool based on big data analysis [J]. Telecommunications Science, 2020, 36(10): 159-171.
[8]	Xiaomin TAN,Ai FANG,Duo JIN,Changjiang LI. Automated anomaly detection of IPTV user experience [J]. Telecommunications Science, 2019, 35(7): 159-164.
[9]	Tianjiao CHEN,Jiang LIU,Tao HUANG. Application of artificial intelligence in network orchestration system [J]. Telecommunications Science, 2019, 35(5): 9-16.
[10]	Xiaosong LIU,Ting HUANG. Application of inspection robot in internet data center intelligent operation and maintenance [J]. Telecommunications Science, 2018, 34(12): 132-137.
[11]	Honghong JIANG,Tao ZHANG,Xinjian ZHAO,Xin QIAN,Tiancheng ZHAO,Lisha GAO. A big data based flow anomaly detection mechanism of electric power information network [J]. Telecommunications Science, 2017, 33(3): 134-141.
[12]	Chuanhuang LI,Cheng CHENG,Xiaoyong YUAN,Lijie CEN,Weiming WANG. Policy conflict detection in software defined network by using deep learning [J]. Telecommunications Science, 2017, 33(11): 27-36.
[13]	Yu Zhang,Lei Dai. Optimizing Network Anomalous Detection Via Heavy Hitter Identification [J]. Telecommunications Science, 2010, 26(3): 46-51.
[14]	Song Yang,Zonglin Li. A Framework of Network-Level OD Flow Anomaly Detection Based on High Aggregate Link Measurement [J]. Telecommunications Science, 2010, 26(10): 121-126.
[15]	Zhou Binbin,Cui Baojiang and Yang Yixian. Research About System-behavior-based Anomaly Detection Technologies [J]. Telecommunications Science, 2009, 25(2): 59-65.