基于混合高斯变分自编码网络的异常检测算法

doi:10.11959/j.issn.1000-0801.2021044

摘要/Abstract

摘要：

异常数据是指偏离大量正常数据点的数据，往往会对各类系统产生负面影响，存在较大风险。异常检测作为一种有效的防护手段，能够检测数据中的异常，为各类系统的正常运转提供重要支撑，具有重要的现实意义。提出了一种基于混合高斯变分自编码网络的异常检测算法，该算法首先使用混合高斯先验构建变分自编码器，对输入数据进行特征提取，然后以混合高斯变分自编码器构建深度支持向量网络，压缩特征空间，并寻找最小超球体分离正常数据和异常数据，通过计算数据特征到超球体中心的欧氏距离衡量数据的异常分数，并以此进行异常检测。最后在基准数据集MNIST和Fashion-MNIST上评估了该算法，平均AUC分别达到了0.954和0.937。实验结果表明，所提出的算法取得了较好的异常检测效果。

关键词: 异常检测, 变分自编码器, 混合高斯分布, 超球体

Abstract:

Anomalous data, which deviates from a large number of normal data, has a negative impact and contains a risk on various systems.Anomaly detection can detect anomalies in the data and provide important support for the normal operation of various systems, which has important practical significance.An anomaly detection algorithm based on Gaussian mixture variational auto encoder network was proposed, in which a variational autoencoder was built to extract the features of the input data based on Gaussian mixture distribution, and using this variational autoencoder to construct a deep support vector network to compress the feature space and find the minimum hyper sphere to separate the normal data and the abnormal data.Anomalies can be detected by the score from the Euclidean distance from the feature of data to the center of the hypersphere.The proposed algorithm was evaluated on the benchmark datasets MNIST and Fashion-MNIST, and the corresponding average AUC are 0.954 and 0.937 respectively.The experimental results show that the proposed algorithm achieves preferable effects.

Key words: anomaly detection, variational autoencoder, Gaussian mixture distribution, hypersphere

中图分类号:

TP393

陈华华, 陈哲, 郭春生, 应娜, 叶学义, 章坚武. 基于混合高斯变分自编码网络的异常检测算法[J]. 电信科学, 2021, 37(4): 54-61.

Huahua CHEN, Zhe CHEN, Chunsheng GUO, Na YING, Xueyi YE, Jianwu ZHANG. Anomaly detection algorithm based on Gaussian mixture variational auto encoder network[J]. Telecommunications Science, 2021, 37(4): 54-61.

图/表 6

图1

表1

图2

表2

MNIST数据集在各种异常检测方法下的实验结果对比"

一般类别	OC-SVM^[21]	IF^[8]	DCAE^[22]	ANOGAN^[23]	本文算法
0	98.6%±0.0%	98.0%±0.0%	97.6%±0.7%	96.6%±1.3%	$99 . 0 %$ ±0.4%
1	99.5%±0.0%	97.3%±0.0%	98.3%±0.6%	99.2%±0.6%	$99 . 7 %$ ±0.1%
2	82.5%±0.1%	88.6%±0.0%	85.4%±2.4%	85.0%±2.9%	$94 . 4 %$ ±0.4%
3	88.1%±0.0%	89.9%±0.0%	86.7%±0.9%	88.7%±2.1%	$94 . 3 %$ ±0.3%
4	$94 . 9 %$ ±0.0%	92.7%±0.0%	86.5%±2.0%	89.4%±1.3%	93.1%±0.5%
5	77.1%±0.0%	85.5%±0.0%	78.2%±2.7%	88.3%±2.9%	$91 . 2 %$ ±0.4%
6	96.5%±0.0%	95.6%±0.0%	94.6%±0.5%	94.7%±2.7%	$98 . 3 %$ ±0.2%
7	93.7%±0.0%	92.0%±0.0%	92.3%±1.0%	$94 . 5 %$ ±1.8%	94.1%±0.3%
8	88.9%±0.0%	89.9%±0.0%	86.5%±1.6%	84.9%±2.1%	$93 . 9 %$ ±0.8%
9	94.1%±0.0%	94.5%±0.0%	90.4%±1.8%	92.4%±1.1%	$95 . 7 %$ ±0.6%
平均值	91.3%±0.0%	92.3%±0.0%	89.7%±1.4%	91.3%±1.9%	$95 . 4 %$ ±0.4%

表2

图3

表3

Fashion-MNIST数据集在各种异常检测方法下的实验结果对比"

一般类别	OC-SVM^[21]	IF^[8]	DCAE^[22]	ANOGAN^[23]	本文算法
0（T恤）	91.4%±0.0%	91.3%±0.0%	95.2%±0.4%	95.8%±0.5%	93.9%±0.3%
1（裤子）	98.2%±0.0%	97.5%±0.0%	98.3%±0.3%	98.5%±0.6%	$98 . 6 %$ ±0.2%
2（套衫）	88.2%±0.1%	87.4%±0.0%	84.3%±0.7%	88.8%±0.9%	$91 . 5 %$ ±0.3%
3（裙子）	91.6%±0.0%	92.2%±0.0%	89.7%±0.5%	88.7%±2.1%	$92 . 4 %$ ±0.7%
4（外套）	89.3%±0.0%	90.8%±0.0%	86.5%±1.3%	90.4%±0.8%	$93 . 5 %$ ±0.3v
5（凉鞋）	89.1%±0.0%	92.6%±0.0%	82.6%±0.7%	90.7%±0.6%	$92 . 5 %$ ±0.5%
6（衬衫）	84.5%±0.1%	80.0%±0.0%	79.3%±0.4%	84.3%±1.4%	$84 . 6 %$ ±1.2%
7（球鞋）	98.5%±0.0%	98.3%±0.0%	92.3%±0.6%	94.5%±0.8%	$98 . 6 %$ ±0.2%
8（皮包）	89.5%±0.0%	88.4%±0.0%	91.7%±0.9%	$94 . 1 %$ ±1.2%	92.1%±1.5%
9（靴子）	97.7%±0.0%	97.6%±0.0%	91.2%±0.5%	92.6%±0.7%	$98 . 8 %$ ±0.1%
平均值	91.8%±0.0%	91.6%±0.0%	89.1%±0.6%	91.6%±1.1%	$93 . 7 %$ ±0.5%

表3

参考文献 23

[1]	HAWKINS D M . Identification of outliers[M]. London: Chapman and Hall, 1980.
[2]	AHMED M , MAHMOOD A N , ISLAM M R . A survey of anomaly detection techniques in financial domain[J]. Future Generation Computer Systems, 2016(55): 278-288.
[3]	张冰涛, 王小鹏, 王履程 ,等. 基于图论的 MANET 入侵检测方法[J]. 电子与信息学报, 2018,40(6): 1446-1452.
	ZHANG B T , WANG X P , WANG L C ,et al. Intrusion detection method for MANET based on graph theory[J]. Journal of Electronics ＆ Information Technology, 2018,40(6): 1446-1452.
[4]	潘雪峰 . 基于 CNN 的手骨异常检测研究与系统实现[D]. 长春:东北师范大学, 2019.
	PAN X F . Research and system implementation of abonormal hand bone detection based on CNN[D]. Changchun:the Dissertation of Northeast Normal University, 2019.
[5]	陆峰峰, 吴浩文, 顾伟 . 基于PI和PSS系统的设备异常状态监控的设计与实现[J]. 工业设计, 2017(4): 167-168.
	LU F F , WU H W , GU W . Design and implementation of equipment abnormal condition monitoring based on PI and PSS system[J]. Indurstrial Design, 2017(4): 167-168.
[6]	吴新宇, 郭会文, 李楠楠 ,等. 基于视频的人群异常事件检测综述[J]. 电子测量与仪器学报, 2014,28(6): 575-584.
	WU X Y , GUO H W , LI N N ,et al. Survey on the video-based abnormal event detection in crowd scenes[J]. Journal of Electronic Measurement and Instrument, 2014,28(6): 575-584.
[7]	SANDER J . LOF:identifying density-based local outliers[J]. ACM SIGMOD Record, 2000,29(2): 93-104.
[8]	LIU F T , TING K M , ZHOU Z H . Isolation forest[C]// Proceedings of 2008 Eighth IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2008: 413-422.
[9]	AN J , CHO S . Variational autoencoder based anomaly detection using reconstruction probability[J]. Special Lecture on IE, 2015(12): 1-18.
[10]	杜辰飞, 高峰, 袁涛 ,等. 面向飞参数据异常检测的 SAE优化方法比较[J]. 信息技术, 2015,39(12): 181-185.
	DU C F , GAO F , YUAN T ,et al. A comparison for optimization methods of SAE in allusion to flight data abnormity detection[J]. Information Technology, 2015,39(12): 181-185.
[11]	LI X , KIRINGA I , YEAP T ,et al. Exploring deep anomaly detection methods based on capsule net[C]// Proceedings of ICML 2019 Workshop on Uncertainty ＆ Robustness in Deep Learning.[S.l.:s.n.], 2019.
[12]	丁建立, 邹云开, 王静 ,等. 基于深度学习的 ADS-B 异常数据检测模型[J]. 航空学报, 2019,40(12): 167-177.
	DING J L , ZOU Y K , WANG J ,et al. ADS-B anomaly data detection model based on deep learning[J]. Acta Aeronautica et Astronautica Sinica, 2019,40(12): 167-177.
[13]	KINGMA D P , WELLING M . Auto-encoding variational bayes[Z]. 2013.
[14]	GUO C , ZHOU J . Variational autoencoder with optimizing Gaussian mixture model priors[J]. IEEE Access, 2020(99): 1.
[15]	HERSHEY J R , OLSEN P A . Approximating the kullback Leibler divergence between Gaussian mixture models[C]// Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2007.
[16]	TAX D M J , DUIN R P W . Support vector data description[J]. Machine Learning, 2004,54(1): 45-66.
[17]	SABOUR S , FROSST N , HINTON G E . Dynamic routing between capsules[C]// Proceedings of Advances in Neural Information Processing Systems. New York:Curran Associates, 2017: 3856-3866.
[18]	TUCKER A L . A reexamination of finite-and infinite-variance distributions as models of daily stock returns[J]. Journal of Business ＆ Economic Statistics, 1992,10(1): 73-81.
[19]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[20]	XIAO H , RASUL K , VOLLGRAF R . Fashion-MNIST:a novel image dataset for benchmarking machine learning algorithms[Z]. 2017.
[21]	MANEVITZ L M , YOUSEF M . One-class SVMs for document classification[J]. Journal of Machine Learning Research, 2001,2(12): 139-154.
[22]	MAKHZANI A , FREY B J . Winner-take-all autoencoders[Z]. 2015.
[23]	SCHLEGL T , SEEB?CK P , WALDSTEIN S M , et al . Unsupervised anomaly detection with generative adversarial networks to guide marker discovery[C]// Proceedings of International Conference on Information Processing in Medical Imaging. Berlin:Springer, 2017: 146-157.

真值	预测值
真值	正常	异常
正常	TP（真正例）	FN（假反例）
异常	FP（假正例）	TN（真反例）