基于钉板分布稀疏变分自编码器的异常检测算法研究

doi:10.11959/j.issn.1000-0801.2022238

Abstract

Abstract:

Anomaly detection remains to be an essential and extensive research branch in data mining due to its widespread use in a wide range of applications.It helps researchers to obtain vital information and make better decisions about data by detecting abnormal data.Considering that sparse coding can get more powerful features and improve the performance of other tasks, an anomaly detection model based on sparse variational autoencoder was proposed.Firstly, the discrete mixed modelspike and slab distribution was used as the prior of variational autoencoder, simulated the sparsity of the space where the hidden variables were located, and obtained the sparse representation of data characteristics.Secondly, combined with the deep support vector network, the feature space was compressed, and the optimal hypersphere was found to discriminate normal data and abnormal data.And then, the abnormal fraction of the data was measured by the Euclidean distance from the data feature to the center of the hypersphere, and then the abnormal detection was carried out.Finally, the algorithm was evaluated on the benchmark datasets MNIST and Fashion-MNIST, and the experimental results show that the proposed algorithm achieves better effects than the state-of-the-art methods.

Key words: anomaly detection, variational autoencoder, spike and slab distribution, deep support vector network

CLC Number:

TP393

Huahua CHEN, Zhe CHEN. Research on anomaly detection algorithm based on sparse variational autoencoder using spike and slab prior[J]. Telecommunications Science, 2022, 38(12): 65-77.

Figures/Tables 10

NORMAL CLASS	OC-SVM^[24]	IF^[9]	DCAE^[25]	ANOGAN^[15]	本文算法
0	$98 . 6 % \pm 0 . 0 %$	98.0%±0.0%	97.6%±0.7%	96.6%±1.3%	98.1%±0.7%
1	99.5%±0.0%	97.3%±0.0%	98.3%±0.6%	99.2%±0.6%	$99 . 6 % \pm 0 . 1 %$
2	82.5%±0.1%	88.6%±0.0%	85.4%±2.4%	85.0%±2.9%	$96 . 5 % \pm 1 . 1 %$
3	88.1%±0.0%	89.9%±0.0%	86.7%±0.9%	88.7%±2.1%	$95 . 4 % \pm 0 . 8 %$
4	94.9%±0.0%	92.7%±0.0%	86.5%±2.0%	89.4%±1.3%	$97 . 6 % \pm 0 . 7 %$
5	77.1%±0.0%	85.5%±0.0%	78.2%±2.7%	88.3%±2.9%	$94 . 6 % \pm 0 . 9 %$
6	96.5%±0.0%	95.6%±0.0%	94.6%±0.5%	94.7%±2.7%	$97 . 7 % \pm 0 . 5 %$
7	93.7%±0.0%	92.0%±0.0%	92.3%±1.0%	94.5%±1.8%	$97 . 0 % \pm 0 . 6 %$
8	88.9%±0.0%	89.9%±0.0%	86.5%±1.6%	84.9%±2.1%	$95 . 3 % \pm 0 . 8 %$
9	94.1%±0.0%	94.5%±0.0%	90.4%±1.8%	92.4%±1.1%	$96 . 3 % \pm 0 . 6 %$
平均值	91.3%±0.0%	92.3%±0.0%	89.7%±1.4%	91.3%±1.9%	$96 . 8 % \pm 0 . 7 %$

NORMAL CLASS	OC-SVM^[24^]	IF^[9]	DCAE^[25]	ANOGAN^[15]	本文算法
0（T-shirt）	91.4%±0.0%	91.3%±0.0%	95.2%±0.4%	$95 . 8 % \pm 0 . 5 %$	95.7%±0.4%
1（Trouser）	98.2%±0.0%	97.5%±0.0%	98.3%±0.3%	98.5%±0.6%	$99 . 0 % \pm 0 . 1 %$
2（Pullover）	88.2%±0.1%	87.4%±0.0%	84.3%±0.7%	88.8%±0.9%	$91 . 2 % \pm 0 . 5 %$
3（Dress）	91.6%±0.0%	92.2%±0.0%	89.7%±0.5%	88.7%±2.1%	$96 . 2 % \pm 0 . 7 %$
4（Coat）	89.3%±0.0%	90.8%±0.0%	86.5%±1.3%	90.4%±0.8%	$92 . 7 % \pm 0 . 3 %$
5（Sandal）	89.1%±0.0%	92.6%±0.0%	82.6%±0.7%	90.7%±0.6%	$93 . 1 % \pm 1 . 6 %$
6（Shirt）	84.5%±0.1%	80.0%±0.0%	79.3%±0.4%	84.3%±1.4%	$84 . 8 % \pm 0 . 9 %$
7（Sneaker）	98.5%±0.0%	98.3%±0.0%	92.3%±0.6%	94.5%±0.8%	$98 . 6 % \pm 0 . 5 %$
8（Bag）	89.5%±0.0%	88.4%±0.0%	91.7%±0.9%	94.1%±1.2%	$97 . 9 % \pm 0 . 7 %$
9（Ankle boot）	97.7%±0.0%	97.6%±0.0%	91.2%±0.5%	92.6%±0.7%	$98 . 8 % \pm 0 . 4 %$
平均值	91.8%±0.0%	91.6%±0.0%	89.1%±0.6%	91.6%±1.1%	$94 . 8 % \pm 0 . 6 %$

对比项	MNIST	Fashion-MNIST
ALOCC DR ^[27]	88.0%	75.3%
ALOCC D ^[27]	82.0%	60.1%
AEC^[28]	89.9%	90.8%
GPND ^[26]	93.2%	90.1%
OCGAN^[29]	97.7%	92.4%
本文算法	$98 . 3 %$	$95 . 2 %$

测试集	10%	20%	30%	40%	50%
GPND^[26]	92.8%	93.2%	93.3%	93.3%	93.3%
GradCon^[30]	93.8%	93.3%	93.5%	93.6%	93.4%
本文算法	$95 . 1 %$	$95 . 3 %$	$95 . 3 %$	$95 . 3 %$	$95 . 2 %$

References 30

[1]	HAWKINS D M . Identification of outliers[M]. London: Chapman and Hall, 1980.
[2]	SHENH D , KURSUN E . Label augmentation via time-based knowledge distillation for financial anomaly detection[EB]. 2021.
[3]	TUFAN E , TEZCAN C , ACARTüRK C , . Anomaly-based intrusion detection by machine learning:acase study on probing attacks to an institutional network[J]. IEEE Access, 9: 50078-50092.
[4]	TSCHUCHNIGM E , GADERMAYR M . Anomaly detection in medical imaging:amini review[EB]. 2021.
[5]	CUI Y J , LIU Z X , LIAN S G ,et al. A survey on unsupervised industrial anomaly detection algorithms[EB]. 2022.
[6]	PATRIKARD R , PARATE M . Anomaly detection using edge computing in video surveillance system:review[J]. International Journal of Multimedia Information Retrieval, 2021,11: 85-110.
[7]	BOGDOLL D , NITSCHE M , Z?LLNERJ M , . Anomaly detection in autonomous driving:asurvey[EB]. 2022.
[8]	J?RG SANDER . LOF:identifying density-based local outliers[J]. Acm Sigmod Record, 2000,29(2): 93-104.
[9]	LIU F T , TING K M , ZHOU Z H . Isolation forest[C]// Proceedings of 2008 Eighth IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2008: 413-422.
[10]	ERFANI S M , RAJASEGARAR S , KARUNASEKERA S ,et al. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning[J]. Pattern Recognition, 2016,58: 121-134.
[11]	LI X Y , KIRINGA I , YEAP T ,et al. Exploring deep anomaly detection methods based on capsule net[C]// Advances in Artificial Intelligence. Switzerland:Springer, 2020: 375-387.
[12]	ZIMMERER D , KOHL S A A , PETERSEN J ,et al. Context-encoding variational autoencoder for unsupervised anomaly detection[EB]. 2018.
[13]	SINGH I , HEMACHANDRA N . Anomaly detection using capsule networks for high-dimensional datasets[EB]. 2021.
[14]	TONOLINI F , JENSEN B S , MURRAY S R . Variational sparse coding[EB]. 2019.
[15]	SCHLEGL T , SEEB?CK P , WALDSTEIN S M ,et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery[C]// Information Processing in Medical Imaging. Switzerland:Springer, 2017: 146-157.
[16]	?KVáRA V , PEVNY T , ?MíDL V , . Are generative deep models for novelty detection truly better?[EB]. 2018.
[17]	SHEIKH A S , SHELTON J A , LüCKE J , . A truncated EM approach for spike-and-slab sparse coding[J]. Journal of Machine Learning Research, 2012,15: 2653-2687.
[18]	COURVILLE A , BERGSTRA J , BENGIO Y ,et al. Unsupervised models of images by spike-and-slab RBMs[EB]. 2011.
[19]	KINGMA D P , WELLING M . Auto-encoding variational Bayes[EB]. 2013.
[20]	TAX D M J , DUIN R P W . Support vector data description[J]. Machine Learning, 2004,54(1): 45-66.
[21]	VAN DER MAATEN L . Learning a parametric embedding by preserving local structure[J]. Journal of Machine Learning Research, 2009,5: 384-391.
[22]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[23]	XIAO H , RASUL K , VOLLGRAF R ,et al. Fashion-MNIST:a novel image dataset for benchmarking machine learning algorithms[EB]. 2017.
[24]	MANEVITZ L M , YOUSEF M . One-class SVMs for document classification[J]. Journal of Machine Learning Research, 2001,2(12): 139-154.
[25]	MAKHZANI A , FREY B J . Winner-take-all autoencoders[C]// Proceedings of Advances in Neural Information Processing Systems(NIPS).Montreal:Neural Information Processing Systems Foundation,Inc. 2015: 2791-2799.
[26]	PIDHORSKYI S , ALMOHSEN R , ADJEROH D A ,et al. Generative probabilistic novelty detection with adversarial autoencoders[C]// Proceedings of Advances in Neural Information Processing Systems(NIPS).Montreal:Neural Information Processing Systems Foundation,Inc. 2018: 6822-6833.
[27]	SABOKROU M , KHALOOEI M , FATHY M ,et al. Adversarially learned one-class classifier for novelty detection[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 3379-3388.
[28]	SAKURADA M , YAIRI T . Anomaly detection using autoencoders with nonlinear dimensionality reduction[C]// MLSDA'14:Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis. 2014: 4-11.
[29]	PERERA P , NALLAPATI R , XIANG B . OCGAN:one-class novelty detection using GANs with constrained latent representations[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 2893-2901.
[30]	KWON G , PRABHUSHANKAR M , TEMEL D ,et al. Back-propagated gradient representations for anomaly detection[C]// Proceedings of the European Conference on Computer Vision (ECCV). Heidelberg:Springer, 2020: 206-226.

Metrics

Recommended 0

No Suggested Reading articles found!

真值		预测值
真值	True		False
True	TP		FN
False	FP		TN

Research on anomaly detection algorithm based on sparse variational autoencoder using spike and slab prior

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 30

Related Articles 10

Metrics

Recommended 0

[1]	Su WANG, Hua LU, Shuo WANG, Lei CAI, Tao HUANG. Research progress of KPI anomaly detection in intelligent operation and maintenance [J]. Telecommunications Science, 2021, 37(5): 42-51.
[2]	Huahua CHEN, Zhe CHEN, Chunsheng GUO, Na YING, Xueyi YE, Jianwu ZHANG. Anomaly detection algorithm based on Gaussian mixture variational auto encoder network [J]. Telecommunications Science, 2021, 37(4): 54-61.
[3]	Xiaomin TAN,Ai FANG,Duo JIN,Changjiang LI. Automated anomaly detection of IPTV user experience [J]. Telecommunications Science, 2019, 35(7): 159-164.
[4]	Honghong JIANG,Tao ZHANG,Xinjian ZHAO,Xin QIAN,Tiancheng ZHAO,Lisha GAO. A big data based flow anomaly detection mechanism of electric power information network [J]. Telecommunications Science, 2017, 33(3): 134-141.
[5]	Chuanhuang LI,Cheng CHENG,Xiaoyong YUAN,Lijie CEN,Weiming WANG. Policy conflict detection in software defined network by using deep learning [J]. Telecommunications Science, 2017, 33(11): 27-36.
[6]	Yu Zhang,Lei Dai. Optimizing Network Anomalous Detection Via Heavy Hitter Identification [J]. Telecommunications Science, 2010, 26(3): 46-51.
[7]	Song Yang,Zonglin Li. A Framework of Network-Level OD Flow Anomaly Detection Based on High Aggregate Link Measurement [J]. Telecommunications Science, 2010, 26(10): 121-126.
[8]	Zhou Binbin,Cui Baojiang and Yang Yixian. Research About System-behavior-based Anomaly Detection Technologies [J]. Telecommunications Science, 2009, 25(2): 59-65.
[9]	Binbin Zhou,Baojiang Cui,Yixian Yang. Research About System-behavior-based Anomaly Detection Technologies [J]. Telecommunications Science, 2009, 1(2): 59-65.
[10]	Weizhou Lu,Shunzheng Yu. Pulsing-based Denial of Service Attack and Defense [J]. Telecommunications Science, 2007, 23(9): 1-76.