基于随机Transformer的多维时间序列异常检测模型

doi:10.11959/j.issn.1000-436x.2023042

摘要/Abstract

摘要：

针对已有基于变分自编码器（VAE）的多维时间序列（MTS）异常检测模型无法在隐空间中传播随机变量间的长时依赖性问题，提出了一种融合 Transformer 编码器和 VAE 的随机 Transformer MTS 异常检测模型（ST-MTS-AD）。在ST-MTS-AD的推断网络中，Transformer编码器产生的当前时刻MTS长时依赖特征和上一时刻随机变量的采样值被输入多层感知器，由此生成当前时刻随机变量的近似后验分布，实现随机变量间的时序依赖。采用门控转换函数（GTF）生成随机变量的先验分布，ST-MTS-AD的生成网络由多层感知器重构MTS各时刻取值分布，该多层感知器的输入为推断网络生成的 MTS 的长时依赖特征和随机变量近似后验采样值。ST-MTS-AD基于变分推断技术学习正常MTS样本集分布，由重构概率对数似然确定MTS异常片段。4个公开数据集上的实验表明，ST-MTS-AD模型比典型相关基线模型的F1分数有明显提升。

关键词: 随机Transformer, 变分自编码器, 多维时间序列, 异常检测

Abstract:

Aiming at the problem that the existing multivariate time series anomaly detection models based on variational autoencoders could not propagate long-term temporal dependencies between stochastic variables in latent space, the stochastic Transformer for MTS anomaly detection (ST-MTS-AD) model which combined Transformer encoder with VAE was proposed.In the inference network of the ST-MTS-AD, the MTS long-term temporal dependent features generated by Transformer encoder and the sampled values of the stochastic variables at the previous moment were inputted into the multilayer perceptron, the approximate posterior distribution of the stochastic variables at the current moment was generated by the multilayer perceptron, and the temporal dependencies between stochastic variables were realized.The gated transition function(GTF) was used to generate the prior distribution of stochastic variables.The generation network of the ST-MTS-AD reconstructed the distribution of the MTS values at each moment by the multilayer perceptron whose input was the MTS long-term temporal dependent features generated by the inference network and the approximate posterior sampling values of stochastic variables.The distribution of normal MTS dataset was learned by the variational inference technology, and the abnormal MTS segment was determined by the log-likelihood of the reconstruction probability.Experiments on four public datasets show that the ST-MTS-AD model significantly improves the F1 score over the typical baseline models.

Key words: stochastic Transformer, variational autoencoder, multivariate time series, anomaly detection

中图分类号:

TP391

霍纬纲, 梁锐, 李永华. 基于随机Transformer的多维时间序列异常检测模型[J]. 通信学报, 2023, 44(2): 94-103.

Weigang HUO, Rui LIANG, Yonghua LI. Anomaly detection model for multivariate time series based on stochastic Transformer[J]. Journal on Communications, 2023, 44(2): 94-103.

图/表 7

图1

图2

图3

图4

表1

表2

ST-MTS-AD模型与6种模型的性能对比"

模型		SMD			SMAP			MSL			SWaT
模型	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
DAGMM	0.879 0	0.832 5	0.855 1	0.800 8	0.999 9	0.889 4	0.676 8	0.998 9	0.807 2	0.997 7	0.687 8	0.814 3
LSTM-VAE	0.974 1	0.804 6	0.881 3	0.997 1	0.888 5	0.939 7	0.995 1	0.847 6	0.915 5	0.998 3	0.684 2	0.812 0
MSCRED	0.682 9	0.986 7	0.807 2	0.784 0	0.990 1	0.878 9	0.883 1	0.999 9	0.937 9	0.999 2	0.677 0	0.807 2
USAD	0.870 9	0.832 5	0.851 3	0.825 6	0.999 9	0.904 4	0.796 9	0.909 9	0.849 1	0.969 6	0.695 6	0.810 1
OmniAnomaly	0.873 8	0.844 0	0.858 6	0.764 8	0.999 9	0.866 7	0.890 2	0.863 7	0.876 7	0.988 6	0.677 0	0.803 6
SDFVAE	0.882 1	0.926 3	0.903 0	0.884 6	0.908 0	0.896 0	0.853 4	0.894 5	0.873 0	0.944 8	0.772 1	$0 . 8497$
ST-MTS-AD	0.893 1	0.977 2	$0 . 9332$	0.998 6	0.936 2	$0 . 9664$	0.993 7	0.970 4	$0 . 9819$	0.975 8	0.728 4	0.834 2

表2

图5

参考文献［20］

［1］	丁小欧, 于晟健, 王沐贤 ,等．基于相关性分析的工业时序数据异常检测［J］．软件学报, 2020,31(3): 726-747．
	DING X O , YU S J , WANG M X ,et al． Anomaly detection on indus-trial time series based on correlation analysis［J］． Journal of Software, 2020,31(3): 726-747．
［2］	SU Y , ZHAO Y J , NIU C H ,et al． Robust anomaly detection for multivariate time series through stochastic recurrent neural network［C］// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining． New York:ACM Press, 2019: 2828-2837．
［3］	LI D , CHEN D , JIN B ,et al． MAD-GAN:multivariate anomaly detection for time series data with generative adversarial networks［C］// International Conference on Artificial Neural Networks． Berlin:Springer, 2019: 214-232．
［4］	黄训华, 张凤斌, 樊好义 ,等．基于多模态对抗学习的无监督时间序列异常检测［J］．计算机研究与发展, 2021,58(8): 1655-1667．
	HUANG X H , ZHANG F B , FAN H Y ,et al． Multimodal adversarial learning based unsupervised time series anomaly detection［J］． Journal of Computer Research and Development, 2021,58(8): 1655-1667．
［5］	段雪源, 付钰, 王坤．基于 VAE-WGAN 的多维时间序列异常检测方法［J］．通信学报, 2022,43(3): 1-13．
	DUAN X Y , FU Y , WANG K ． Multi-dimensional time series anomaly detection method based on VAE-WGAN［J］． Journal on Communica-tions, 2022,43(3): 1-13．
［6］	ZONG B , SONG Q , MIN M R ,et al． Deep autoencoding Gaussian mixture model for unsupervised anomaly detection［C］// International Conference on Learning Representations． Piscataway:IEEE Press, 2018: 1-19．
［7］	DENG L W , CHEN X H , ZHAO Y ,et al． HIFI:anomaly detection for multivariate time series with high-order feature interactions［C］// International Conference on Database Systems for Advanced Applications． Berlin:Springer, 2021: 641-649．
［8］	PARK D , HOSHI Y , KEMP C C ． A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder［J］． IEEE Robotics and Automation Letters, 2018,3(3): 1544-1551．
［9］	DAI L , LIN T , LIU C ,et al． SDFVAE:static and dynamic factorized VAE for anomaly detection of multivariate CDN KPIs［C］// Proceedings of the Web Conference 2021． New York:ACM Press, 2021: 3076-3086．
［10］	LI L Y , YAN J C , WANG H Y ,et al． Anomaly detection of time series with smoothness-inducing sequential variational auto-encoder［J］． IEEE Transactions on Neural Networks and Learning Systems, 2021,32(3): 1177-1191．
［11］	AHMAD S , LAVIN A , PURDY S ,et al． Unsupervised real-time anomaly detection for streaming data［J］． Neurocomputing, 2017,262: 134-147．
［12］	HUNDMAN K , CONSTANTINOU V , LAPORTE C ,et al． Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining． New York:ACM Press, 2018: 387-395．
［13］	ZHANG Y X , CHEN Y Q , WANG J D ,et al． Unsupervised deep anomaly detection for multi-sensor time-series signals［J］． IEEE Transactions on Knowledge and Data Engineering, 2023,35(2): 2118-2132．
［14］	MALHOTRA P , RAMAKRISHNAN A , ANAND G ,et al． LSTM-based encoder-decoder for multi-sensor anomaly detection［J］． arXiv Preprint,arXiv:1607．00148, 2016．
［15］	ZHANG C X , SONG D J , CHEN Y C ,et al． A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data［J］． Proceedings of the AAAI Conference on Artificial Intelligence, 2019,33(1): 1409-1416．
［16］	AUDIBERT J , MICHIARDI P , GUYARD F ,et al． USAD:unsupervised anomaly detection on multivariate time series［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining． New York:ACM Press, 2020: 3395-3404．
［17］	KRISHNAN R G , SHALIT U , SONTAG D ． Structured inference networks for nonlinear state space models［C］// Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence． New York:ACM Press, 2017: 2101-2109．
［18］	CHUNG J , KASTNER K , DINH L ,et al． A recurrent latent variable model for sequential data［J］． Advances in Neural Information Processing Systems, 2015,28: 2962-2970．
［19］	FRACCARO M , SNDERBY S K , PAQUET U ,et al． Sequential neural models with stochastic layers［J］． Advances in Neural Information Processing Systems, 2016,29: 2199-2207．
［20］	ZERVEAS G , JAYARAMAN S , PATEL D ,et al． A transformer-based framework for multivariate time series representation learning［C］// Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery ＆ Data Mining． New York:ACM Press, 2021: 2114-2124．

数据集	训练数据长度	测试数据长度	维度
SMD	708 405	708 420	38
SMAP	135 183	427 617	25
MSL	58 317	73 729	55
SWaT	99 360	89 984	51

[1]	王一丰, 郭渊博, 陈庆礼, 方晨, 林韧昊, 周永良, 马佳利. 基于对比增量学习的细粒度恶意流量分类方法[J]. 通信学报, 2023, 44(3): 1-11.
[2]	王延文, 雷为民, 张伟, 孟欢, 陈新怡, 叶文慧, 景庆阳. 基于生成模型的视频图像重建方法综述[J]. 通信学报, 2022, 43(9): 194-208.
[3]	廖建新, 付霄元, 戚琦, 王敬宇, 孙海峰. 6G-ADM：基于知识空间的6G网络管控体系[J]. 通信学报, 2022, 43(6): 3-15.
[4]	段雪源, 付钰, 王坤. 基于VAE-WGAN的多维时间序列异常检测方法[J]. 通信学报, 2022, 43(3): 1-13.
[5]	吴平, 常朝稳, 左志斌, 马莹莹. 基于地址重载的SDN分组转发验证[J]. 通信学报, 2022, 43(3): 88-100.
[6]	孙海丽, 龙翔, 韩兰胜, 黄炎, 李清波. 工业物联网异常检测技术综述[J]. 通信学报, 2022, 43(3): 196-210.
[7]	陈卓, 朱淼, 杜军威. 基于多视角图神经网络的欺诈检测算法[J]. 通信学报, 2022, 43(11): 225-232.
[8]	王一丰, 郭渊博, 陈庆礼, 方晨, 林韧昊. 基于对比学习的细粒度未知恶意流量分类方法[J]. 通信学报, 2022, 43(10): 12-25.
[9]	段雪源, 付钰, 王坤, 刘涛涛, 李彬. 基于多尺度特征的网络流量异常检测方法[J]. 通信学报, 2022, 43(10): 65-76.
[10]	朱会娟, 陈锦富, 李致远, 殷尚男. 基于多特征自适应融合的区块链异常交易检测方法[J]. 通信学报, 2021, 42(5): 41-50.
[11]	陈铁明,金成强,吕明琪,朱添田. 基于样本增强的网络恶意流量智能检测方法[J]. 通信学报, 2020, 41(6): 128-138.
[12]	戚琦,申润业,王敬宇. GAD：基于拓扑感知的时间序列异常检测[J]. 通信学报, 2020, 41(6): 152-160.
[13]	杨晓晖,张圣昌. 基于多粒度级联孤立森林算法的异常检测模型[J]. 通信学报, 2019, 40(8): 133-142.
[14]	李佳,云晓春,李书豪,张永铮,谢江,方方. 基于混合结构深度神经网络的HTTP恶意流量检测方法[J]. 通信学报, 2019, 40(1): 24-33.
[15]	吴志军,张景安,岳猛,张才峰. 基于联合特征的LDoS攻击检测方法[J]. 通信学报, 2017, 38(5): 19-30.