基于钉板分布稀疏变分自编码器的异常检测算法研究

doi:10.11959/j.issn.1000-0801.2022238

摘要/Abstract

摘要：

异常检测由于其广泛的应用一直是数据挖掘中一个重要的研究分支，它有助于研究人员获得重要的信息进而对数据做出更好的决策。提出了一种基于钉板分布稀疏变分自编码器的异常检测模型。首先，使用离散-连续混合模型钉板分布作为变分自编码器的先验，模拟隐变量所在空间的稀疏性，得到数据特征的稀疏表示；其次，以所提出的自编码器构建深度支持向量网络，对特征空间进行压缩，并采用最优超球体区分正常数据和异常数据；再次，以数据特征和超球体中心之间的欧氏距离完成异常检测；最后，在基准数据集MNIST （modifiednational institute of standards and technology database）和Fashion-MNIST上的实验评估表明，与现存的异常检测算法相比，本文所提出的算法具有更好的检测效果。

关键词: 异常检测, 变分自编码器, 钉板分布, 深度支持向量网络

Abstract:

Anomaly detection remains to be an essential and extensive research branch in data mining due to its widespread use in a wide range of applications.It helps researchers to obtain vital information and make better decisions about data by detecting abnormal data.Considering that sparse coding can get more powerful features and improve the performance of other tasks, an anomaly detection model based on sparse variational autoencoder was proposed.Firstly, the discrete mixed modelspike and slab distribution was used as the prior of variational autoencoder, simulated the sparsity of the space where the hidden variables were located, and obtained the sparse representation of data characteristics.Secondly, combined with the deep support vector network, the feature space was compressed, and the optimal hypersphere was found to discriminate normal data and abnormal data.And then, the abnormal fraction of the data was measured by the Euclidean distance from the data feature to the center of the hypersphere, and then the abnormal detection was carried out.Finally, the algorithm was evaluated on the benchmark datasets MNIST and Fashion-MNIST, and the experimental results show that the proposed algorithm achieves better effects than the state-of-the-art methods.

Key words: anomaly detection, variational autoencoder, spike and slab distribution, deep support vector network

中图分类号:

TP393

陈华华, 陈哲. 基于钉板分布稀疏变分自编码器的异常检测算法研究[J]. 电信科学, 2022, 38(12): 65-77.

Huahua CHEN, Zhe CHEN. Research on anomaly detection algorithm based on sparse variational autoencoder using spike and slab prior[J]. Telecommunications Science, 2022, 38(12): 65-77.

图/表 10

图1

表1

图2

图3

图4

图5

表2

MNIST数据集在各种异常检测方法下的实验结果对比"

NORMAL CLASS	OC-SVM^[24]	IF^[9]	DCAE^[25]	ANOGAN^[15]	本文算法
0	$98 . 6 % \pm 0 . 0 %$	98.0%±0.0%	97.6%±0.7%	96.6%±1.3%	98.1%±0.7%
1	99.5%±0.0%	97.3%±0.0%	98.3%±0.6%	99.2%±0.6%	$99 . 6 % \pm 0 . 1 %$
2	82.5%±0.1%	88.6%±0.0%	85.4%±2.4%	85.0%±2.9%	$96 . 5 % \pm 1 . 1 %$
3	88.1%±0.0%	89.9%±0.0%	86.7%±0.9%	88.7%±2.1%	$95 . 4 % \pm 0 . 8 %$
4	94.9%±0.0%	92.7%±0.0%	86.5%±2.0%	89.4%±1.3%	$97 . 6 % \pm 0 . 7 %$
5	77.1%±0.0%	85.5%±0.0%	78.2%±2.7%	88.3%±2.9%	$94 . 6 % \pm 0 . 9 %$
6	96.5%±0.0%	95.6%±0.0%	94.6%±0.5%	94.7%±2.7%	$97 . 7 % \pm 0 . 5 %$
7	93.7%±0.0%	92.0%±0.0%	92.3%±1.0%	94.5%±1.8%	$97 . 0 % \pm 0 . 6 %$
8	88.9%±0.0%	89.9%±0.0%	86.5%±1.6%	84.9%±2.1%	$95 . 3 % \pm 0 . 8 %$
9	94.1%±0.0%	94.5%±0.0%	90.4%±1.8%	92.4%±1.1%	$96 . 3 % \pm 0 . 6 %$
平均值	91.3%±0.0%	92.3%±0.0%	89.7%±1.4%	91.3%±1.9%	$96 . 8 % \pm 0 . 7 %$

表2

表3

Fashion-MNIST数据集在各种异常检测方法下的实验结果对比"

NORMAL CLASS	OC-SVM^[24^]	IF^[9]	DCAE^[25]	ANOGAN^[15]	本文算法
0（T-shirt）	91.4%±0.0%	91.3%±0.0%	95.2%±0.4%	$95 . 8 % \pm 0 . 5 %$	95.7%±0.4%
1（Trouser）	98.2%±0.0%	97.5%±0.0%	98.3%±0.3%	98.5%±0.6%	$99 . 0 % \pm 0 . 1 %$
2（Pullover）	88.2%±0.1%	87.4%±0.0%	84.3%±0.7%	88.8%±0.9%	$91 . 2 % \pm 0 . 5 %$
3（Dress）	91.6%±0.0%	92.2%±0.0%	89.7%±0.5%	88.7%±2.1%	$96 . 2 % \pm 0 . 7 %$
4（Coat）	89.3%±0.0%	90.8%±0.0%	86.5%±1.3%	90.4%±0.8%	$92 . 7 % \pm 0 . 3 %$
5（Sandal）	89.1%±0.0%	92.6%±0.0%	82.6%±0.7%	90.7%±0.6%	$93 . 1 % \pm 1 . 6 %$
6（Shirt）	84.5%±0.1%	80.0%±0.0%	79.3%±0.4%	84.3%±1.4%	$84 . 8 % \pm 0 . 9 %$
7（Sneaker）	98.5%±0.0%	98.3%±0.0%	92.3%±0.6%	94.5%±0.8%	$98 . 6 % \pm 0 . 5 %$
8（Bag）	89.5%±0.0%	88.4%±0.0%	91.7%±0.9%	94.1%±1.2%	$97 . 9 % \pm 0 . 7 %$
9（Ankle boot）	97.7%±0.0%	97.6%±0.0%	91.2%±0.5%	92.6%±0.7%	$98 . 8 % \pm 0 . 4 %$
平均值	91.8%±0.0%	91.6%±0.0%	89.1%±0.6%	91.6%±1.1%	$94 . 8 % \pm 0 . 6 %$

表3

表4

80%正常样本作为训练的模型实验结果对比"

对比项	MNIST	Fashion-MNIST
ALOCC DR ^[27]	88.0%	75.3%
ALOCC D ^[27]	82.0%	60.1%
AEC^[28]	89.9%	90.8%
GPND ^[26]	93.2%	90.1%
OCGAN^[29]	97.7%	92.4%
本文算法	$98 . 3 %$	$95 . 2 %$

表4

表5

Fashion-MNIST集中异常样本在测试集不同占比下的实验结果对比"

测试集	10%	20%	30%	40%	50%
GPND^[26]	92.8%	93.2%	93.3%	93.3%	93.3%
GradCon^[30]	93.8%	93.3%	93.5%	93.6%	93.4%
本文算法	$95 . 1 %$	$95 . 3 %$	$95 . 3 %$	$95 . 3 %$	$95 . 2 %$

表5

参考文献 30

[1]	HAWKINS D M . Identification of outliers[M]. London: Chapman and Hall, 1980.
[2]	SHENH D , KURSUN E . Label augmentation via time-based knowledge distillation for financial anomaly detection[EB]. 2021.
[3]	TUFAN E , TEZCAN C , ACARTüRK C , . Anomaly-based intrusion detection by machine learning:acase study on probing attacks to an institutional network[J]. IEEE Access, 9: 50078-50092.
[4]	TSCHUCHNIGM E , GADERMAYR M . Anomaly detection in medical imaging:amini review[EB]. 2021.
[5]	CUI Y J , LIU Z X , LIAN S G ,et al. A survey on unsupervised industrial anomaly detection algorithms[EB]. 2022.
[6]	PATRIKARD R , PARATE M . Anomaly detection using edge computing in video surveillance system:review[J]. International Journal of Multimedia Information Retrieval, 2021,11: 85-110.
[7]	BOGDOLL D , NITSCHE M , Z?LLNERJ M , . Anomaly detection in autonomous driving:asurvey[EB]. 2022.
[8]	J?RG SANDER . LOF:identifying density-based local outliers[J]. Acm Sigmod Record, 2000,29(2): 93-104.
[9]	LIU F T , TING K M , ZHOU Z H . Isolation forest[C]// Proceedings of 2008 Eighth IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2008: 413-422.
[10]	ERFANI S M , RAJASEGARAR S , KARUNASEKERA S ,et al. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning[J]. Pattern Recognition, 2016,58: 121-134.
[11]	LI X Y , KIRINGA I , YEAP T ,et al. Exploring deep anomaly detection methods based on capsule net[C]// Advances in Artificial Intelligence. Switzerland:Springer, 2020: 375-387.
[12]	ZIMMERER D , KOHL S A A , PETERSEN J ,et al. Context-encoding variational autoencoder for unsupervised anomaly detection[EB]. 2018.
[13]	SINGH I , HEMACHANDRA N . Anomaly detection using capsule networks for high-dimensional datasets[EB]. 2021.
[14]	TONOLINI F , JENSEN B S , MURRAY S R . Variational sparse coding[EB]. 2019.
[15]	SCHLEGL T , SEEB?CK P , WALDSTEIN S M ,et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery[C]// Information Processing in Medical Imaging. Switzerland:Springer, 2017: 146-157.
[16]	?KVáRA V , PEVNY T , ?MíDL V , . Are generative deep models for novelty detection truly better?[EB]. 2018.
[17]	SHEIKH A S , SHELTON J A , LüCKE J , . A truncated EM approach for spike-and-slab sparse coding[J]. Journal of Machine Learning Research, 2012,15: 2653-2687.
[18]	COURVILLE A , BERGSTRA J , BENGIO Y ,et al. Unsupervised models of images by spike-and-slab RBMs[EB]. 2011.
[19]	KINGMA D P , WELLING M . Auto-encoding variational Bayes[EB]. 2013.
[20]	TAX D M J , DUIN R P W . Support vector data description[J]. Machine Learning, 2004,54(1): 45-66.
[21]	VAN DER MAATEN L . Learning a parametric embedding by preserving local structure[J]. Journal of Machine Learning Research, 2009,5: 384-391.
[22]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[23]	XIAO H , RASUL K , VOLLGRAF R ,et al. Fashion-MNIST:a novel image dataset for benchmarking machine learning algorithms[EB]. 2017.
[24]	MANEVITZ L M , YOUSEF M . One-class SVMs for document classification[J]. Journal of Machine Learning Research, 2001,2(12): 139-154.
[25]	MAKHZANI A , FREY B J . Winner-take-all autoencoders[C]// Proceedings of Advances in Neural Information Processing Systems(NIPS).Montreal:Neural Information Processing Systems Foundation,Inc. 2015: 2791-2799.
[26]	PIDHORSKYI S , ALMOHSEN R , ADJEROH D A ,et al. Generative probabilistic novelty detection with adversarial autoencoders[C]// Proceedings of Advances in Neural Information Processing Systems(NIPS).Montreal:Neural Information Processing Systems Foundation,Inc. 2018: 6822-6833.
[27]	SABOKROU M , KHALOOEI M , FATHY M ,et al. Adversarially learned one-class classifier for novelty detection[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 3379-3388.
[28]	SAKURADA M , YAIRI T . Anomaly detection using autoencoders with nonlinear dimensionality reduction[C]// MLSDA'14:Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis. 2014: 4-11.
[29]	PERERA P , NALLAPATI R , XIANG B . OCGAN:one-class novelty detection using GANs with constrained latent representations[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 2893-2901.
[30]	KWON G , PRABHUSHANKAR M , TEMEL D ,et al. Back-propagated gradient representations for anomaly detection[C]// Proceedings of the European Conference on Computer Vision (ECCV). Heidelberg:Springer, 2020: 206-226.

真值		预测值
真值	True		False
True	TP		FN
False	FP		TN