Abstract:

Aiming at the problem that the existing multivariate time series anomaly detection models based on variational autoencoders could not propagate long-term temporal dependencies between stochastic variables in latent space, the stochastic Transformer for MTS anomaly detection (ST-MTS-AD) model which combined Transformer encoder with VAE was proposed.In the inference network of the ST-MTS-AD, the MTS long-term temporal dependent features generated by Transformer encoder and the sampled values of the stochastic variables at the previous moment were inputted into the multilayer perceptron, the approximate posterior distribution of the stochastic variables at the current moment was generated by the multilayer perceptron, and the temporal dependencies between stochastic variables were realized.The gated transition function(GTF) was used to generate the prior distribution of stochastic variables.The generation network of the ST-MTS-AD reconstructed the distribution of the MTS values at each moment by the multilayer perceptron whose input was the MTS long-term temporal dependent features generated by the inference network and the approximate posterior sampling values of stochastic variables.The distribution of normal MTS dataset was learned by the variational inference technology, and the abnormal MTS segment was determined by the log-likelihood of the reconstruction probability.Experiments on four public datasets show that the ST-MTS-AD model significantly improves the F1 score over the typical baseline models.

Key words: stochastic Transformer, variational autoencoder, multivariate time series, anomaly detection