基于观测数据的时间序列因果推断综述

doi:10.11959/j.issn.2096-0271.2022059

摘要/Abstract

摘要：

数据存储量的扩大和计算能力的提升，为基于观测数据推断时间序列的因果关系开辟了新途径。在时间序列因果推断的基本性质和研究现状的基础上，系统梳理了5种基于观测数据的时间序列因果推断方法，即Granger因果分析方法、基于信息论的方法、因果网络结构学习算法、基于结构因果模型的方法和基于非线性状态空间模型的方法。然后，根据不同应用场景的数据特点，结合方法的功能和适配性，对基于观测数据的时间序列因果推断方法在经济金融、医疗和生物学、地球系统科学和其他工程领域的典型应用进行了简要介绍。最后，结合时间序列因果推断的重难点问题，比较5种方法的优缺点，分析下一步研究重点，展望未来的研究方向。

关键词:

时间序列, 因果推断, Granger因果分析, 信息熵, 贝叶斯网络, 结构因果模型, 非线性状态空间模型

Abstract:

With the increase of data storage and the improvement of computing power, using observational data to infer time series causality has become a novel approach. Based on the properties and research status of time series causal inference, five observational data-based methods are induced, including Granger causal analysis, information theory-based method, causal network structure learning algorithm, structural causal model-based method and method based on nonlinear state-space model. Then we briefly introduced typical applications in economics and finance, medical science and biology, earth system science and other engineering fields. Further, we compared the advantages and disadvantages and analyzed the ways for improvement of the five methods according to the focus and difficulties of time series causal inference. Finally, we looked into the future research directions.

Key words:

"> time series, causal inference, Granger causal analysis, information entropy, Bayesian network, structural causal model, nonlinear state space model

曾泽凡, 陈思雅, 龙洗, 金光.

基于观测数据的时间序列因果推断综述 [J]. 大数据, doi: 10.11959/j.issn.2096-0271.2022059.

ZENG Zefan, CHEN Siya, LONG Xi, JIN Guang.

Overview of Observational Data-Based Time Series [J]. Big Data Research, doi: 10.11959/j.issn.2096-0271.2022059.

参考文献

[1] BROCKWELL P J , DAVIS R A , BERGER J O , et al. Time Series: Theory and Methods[M]. Berlin, Springer-Verlag, 2015: 2-35.

[2] 韩敏,任伟杰,李柏松,冯守渤.混沌时间序列分析与预测研究综述[J].信息与控制,2020,49(01):24-35.

HAN M, REN W ,LI B, et al. A Review of Chaotic Time Series Analysis and Forecasting[J].Information and Control (Chinese),2020,49(01):24-35.

[3] PEARL J , GLYMOUR M , JEWELL N P . Causal Inference in Statistics: A Primer [M]. New Jersey, John Wiley & Sons, 2016: 22-58.

[4] 任伟杰,韩敏.多元时间序列因果关系分析研究综述[J].自动化学报,2021,47(01):64-78.

REN W, HAN M. A review of causality analysis of multiple time series[J].Acta Automation Sinica,2021,47(01):64-78.

[5] PARUNAK H . Elements of causal inference: foundations and learning algorithms[J]. Computing reviews, 2018, 59(11):588-589.

[6] PEARL, J, MACKENZIE, D. The Book of Why: The New Science of Cause and Effect [M]. London, Allen Lane Press, 2018: 126-155.

[7] YAO L , CHU Z , LI S , et al. A Survey on Causal Inference[J]. ACM Transactions on Knowledge Discovery from Data, 2021, 15(5):1-46.

[8] SCHÖLKOPF B, LOCATELLO F, BAUER S, et. al. Toward Causal Representation Learning[C]//in Proceedings of the IEEE, 2021, 109(5), 612-634.

[9] GRANGER C , WIENER J. Investigating causal relations by econometric models and cross-spectral methods[J], Econometrica, 1969, 37: 424-438.

[10] N. WIENER. The Theory of Prediction. I n Modern Mathematics for Engineers[M].New York, NY, McGraw-Hill, 1956: 368-398.

[11] SETH A K, BARRETT A B, BARNETT L. Granger causality analysis in neuroscience and neuroimaging[J]. The Journal of Neuroscience, 2015, 35(8): 3293−3297.

[12] AUGUSTINE C. ARIZE. Determinants of Income Velocity in the United Kingdom: Multivariate Granger Causality [J].The American Economist,1993, 37(2): 40–45.

[13] GEWEKE, JOHN. Measurement of Linear Dependence and Feedback between Multiple Time Series[J]. Publications of the American Statistical Association, 1982, 77(378):304-313.

[14] CRAIG, HIEMSTRA, JOHNATHAN, et al. Testing for Linear and Nonlinear Granger Causality in the Stock Price-Volume Relation[J]. The Journal of Finance, 1994, 49(5):1639-1664.

[15] ANCONA N , MARINAZZO D , STRAMAGLIA S . Radial basis function approach to nonlinear Granger causality of time series[J]. Phys Rev E Stat Nonlin Soft Matter Phys, 2004, 70(5 Pt 2):056221.

[16] WEI L , D MARINAZZO, PA N Z , et al. Kernel Granger Causality Mapping Effective Connectivity on fMRI Data[J]. 2009, 28(11):1825-1835.

[17] ZHOU Y , KANG Z , LIN Z , et al. Causal analysis for non-stationary time series in sensor-rich smart buildings[C]// Proceedings of IEEE International Conference on Automation Science & Engineering. IEEE, 2013: 593-598.

[18] SCHAECK T , MUMA M , FENG M , et al. Robust Nonlinear Causality Analysis of Non-Stationary Multivariate Physiological Time Series[J]. IEEE Transactions on Biomedical Engineering, 2017, 65(6):1213-1225.

[19] WEI CHEN, JIBIN CHEN , RUICHU CAI , et al. Learning granger causality for non-stationary Hawkes processes[J]. Neurocomputing, 2022, 468:22-32.

[20] SCHINDLER K . Granger Lasso Causal Models in Higher Dimensions - Application to Gene Expression Regulatory Networks[C]// Proceedings of Machine Learning and Knowledge Discovery in Databases European Conference. 2013: 1-14.

[21] YANG G , WANG L , WANG X . Reconstruction of Complex Directional Networks with Group Lasso Nonlinear Conditional Granger Causality[J]. Scientific Reports, 2017, 7(1):2991.

[22] BAHADORI, T., Y. LIU, Y. An Examination of Large-Scale Granger Causality Inference[C]// Proceedings of SIAM Conference on Data Mining (SDM’13),2013: 2301-2309.

[23] SCHREIBER, THOMAS. Measuring Information Transfer[J]. Physical Review Letters, 2000, 85(2):461-464.

[24] BOSSOMAIER T , BARNETT L , M HARRÉ, et al. Transfer Entropy[M]. Springer International Publishing, 2016.

[25] KRASKOV A , H STOEGBAUER, GRASSBERGER P . Estimating Mutual Information[J].Physical Review E, 2004, 69:066138.

[26] STANIEK M , LEHNERTZ K . Symbolic Transfer Entropy[J]. Physical Review Letters, 2008, 100(15):158101.

[27] KUGIUMTZIS D . Partial transfer entropy on rank vectors[J]. European Physical Journal Special Topics, 2013, 222(2):401-420.

[28] RASHIDI B , ZHAO Q . Autonomous Root-Cause Fault Diagnosis Using Symbolic Dynamic Based Causality Analysis[J]. Neurocomputing, 2020, 401(3):10-27.

[29] ZENG Z, JIN G, XU C, CHEN S, ZHANG L. Spacecraft Telemetry Anomaly Detection Based on Parametric Causality and Double-Criteria Drift Streaming Peaks over Threshold[J]. Applied Sciences, 2022; 12(4):1803.

[30] SUN J , BOLLT E M . Causation Entropy Identifies Indirect Influences, Dominance of Neighbors and Anticipatory Couplings[J]. Physica D Nonlinear Phenomena, 2014, 267:49-57.

[31] 郝志峰, 谢蔚涛, 蔡瑞初,等. 基于因果强度的时序因果关系发现算法[J]. 计算机工程与设计, 2017, 38(1):6.

HAO ZF, XIE WT, CAI RC, et al. Time-series causal relationship discovery algorithm based on causal strength. Computer Engineering and Design, 2017, 38(1):6.

[32] LIZIER J , RUBINOV M . Multivariate construction of effective computational networks from observational data[J]. Avian Diseases, 2012, 30(1):1-2.

[33] CHEN S , JIN G , MA X . Detection and Analysis of Real-time Anomalies in Large-Scale Complex System[J]. Measurement, 2021(2):109929.

[34] BARNETT L , BARRETT A B , SETH A K . Granger causality and transfer entropy are equivalent for Gaussian variables[J]. Physical Review Letters, 2009, 103.

[35] NEAPOLITAN R E . Learning Bayesian Networks[M].New Jersey, Prentice Hall, 2003: 1-33.

[36] 张连文，郭海鹏. 贝叶斯网引论[M]. 北京：科学出版社, 2006.

Zhang L W, Guo H P. Introduction to Bayesian Networks [M]. Beijing: Science Press, 2006.

[37] 蔡瑞初，郝志峰.大数据中的因果关系发现[M].北京：科学出版社，2018: 22-49.

CAI R H, HAO Z F. Causal Relationship Discovery in Big Data . Beijing: Science Press, 2018: 22-49.

[38] SPIRTES P, GLYMOUR C , SCHEINES R. Causation, Prediction, and Search[M]. Cambridge, MIT Press, 2000.

[39] MARKUS KALISCH, PETER BÜHLMANN. Estimating High-Dimensional Directed Acyclic Graphs with the PC-Algorithm[J]. Journal of Machine Learning Research, 2007, 8:613-636.

[40] VERMA T, PEARL J. Equivalence and synthesis of causal models[C]// Proceedings of the 6th Conference on Uncertainty in Artificial Intelligence. Cambridge, UK, 1990: 255-268.

[41] HUND L , SCHROEDER B . A causal perspective on reliability assessment[J]. Reliability Engineering and System Safety, 2019, 195:106678.

[42] CHICKERING D M . Learning Equivalence Classes of Bayesian Networks Structures[J]. arxiv: 1302/1302.3566.pdf, 2013.

[43] TSAMARDINOS I, BROWN L E, ALIFERIS C F. The max-min hill-climbing Bayesian network structure learning algorithm[J]. Machine Learning, 2006, 65(1): 31-78.

[44] TSAMARDINOS I, ALIFERIS C F, STATNIKOV A. Time and sample efficient discovery of Markov blankets and direct causal relations[C]//Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington, USA, 2003: 673-678.

[45] HUANG B , ZHANG K , ZHANG J , et al. Causal Discovery from Heterogeneous/Nonstationary Data[J]. arXiv:1903.01672v5[cs.LG], 2019.

[46] SPIRTES P , MEEK C , RICHARDSON T . An Algorithm for causal inference in the presence of latent variables and selection bias[J]. Computation Causation & Discovery, 2016:211-252.

[47] COLOMBO D, MAATHUIS M H, KALISCH M, et al. Learning high-dimensional directed acyclic graphs with latent and selection variables[J]. The Annals of Statistics, 2012, 40(1): 294-321.

[48] OGARRIO J M, SPIRTES P, RAMSEY J. A hybrid causal search algorithm for latent variable models[C]//Proceedings of Conference on Probabilistic Graphical Models. 2016: 368-379.

[49] KUMMERFELD E, RAMSEY J, YANG R, et al. Causal clustering for 2-factor measurement models//Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2014). Nancy, France, 2014: 34-49.

[50] RUNGE J . Causal network reconstruction from time series: From theoretical assumptions to practical estimation[J]. Chaos, 2018, 28(7):075310.

[51] RUNGE J , SEJDINOVIC D , FLAXMAN S . Detecting causal associations in large nonlinear time series datasets[J]. arXiv:1702.07007v2 [stat.ME],2017.

[52] RUNGE J . Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets[J]. arXiv:2003.03685v2 [stat.ME], 2020.

[53] SHIMIZU S, HOYER P O, HYVÄRINEN A, et al. A linear non-Gaussian acyclic model for causal discovery[J]. Journal of Machine Learning Research, 2006, 7: 2003-2030.

[54] RUNGE J , BATHIANY S , BOLT E , et al. Inferring causation from time series in Earth system sciences[J]. Nature Communications, 2019, 10(1).

[55] AAPO HYVÄRINEN, HOYER P O , INKI M . Topographic ICA as a Model of Natural Image Statistics[C]// Proceedings of IEEE International Workshop on Biologically Motivated Computer Vision. Springer-Verlag, 2000: 83-88.

[56] SHIMIZU S, INAZUMI T, SOGAWA Y, et al. DirectLiNGAM: A direct method for learning a linear non-gaussian structural equation model[J]. Journal of Machine Learning Research, 2011, 12(2): 1225-1248.

[57] HOYER P O, JANZING D, MOOIJ J M, et al. Nonlinear causal discovery with additive noise models[C]//Proceedings of the 23rd Annual Conference on Neural Information Processing Systems (NIPS 2009). Vancouver, Canada, 2009: 689-696.

[58] M. MOOIJ, D. JANZING, J. PETERS, AND B. SCHÖLKOPF. Regression by dependence minimization and its application to causal inference[C]// Proceedings of the 26^thInternational Conference on Machine Learning (ICML), 2009,: 745–752.

[59] NOWZOHOUR , P. BÜHLMANN. Score-based causal learning in additive noise models[J]. Statistics, 2016, 50(3):471–485.

[60] ZHANG K, HYVÄRINEN A. On the identifiability of the post-nonlinear causal model[C]//Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009). Montreal, Canada, 2009: 647-655.

[61] TASHIRO T, SHIMIZU S, HYVÄRINEN A, et al. ParceLiNGAM: A causal ordering method robust against latent confounders[J]. Neural Computation, 2014, 26(1): 57-83.

[62] HOYER P O, SHIMIZU S, KERMINEN A J, et al. Estimation of causal effects using linear non-Gaussian causal models with hidden variables[J]. International Journal of Approximate Reasoning, 2008, 49(2): 362-378.

[63] CAI R, XIE F, GLYMOUR C, et al. Triad Constraints for Learning Causal Structure of Latent Variables[C]//Advances in Neural Information Processing Systems. 2019: 12863-12872.

[64] JANZING D, MOOIJ J, ZHANG KUN, et al. Information-geometric approach to inferring causal directions[J]. Artificial Intelligence, 2012, 182: 1-31.

[65] XIE F , CAI R , ZENG Y , et al. An Efficient Entropy-Based Causal Discovery Method for Linear Structural Equation Models With IID Noise Variables[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, PP(99):1-14.

[66] SUGIHARA G , MAY R , YE H , et al. Detecting Causality in Complex Ecosystems[J]. Science, 2012, 338(6106):496-500.

[67] F TAKENS. Dynamical Systems and Turbulence [M]. NewYork, Springer-Verlag, 1981:67-88.

[68] ARNHOLD J , GRASSBERGER P , LEHNERTZ K , et al. A Robust Method for Detecting Interdependences: Application to Intracranially Recorded EEG[J]. Physica D-nonlinear Phenomena, 1999, 134(4):419-430.

[69] QUIROGA R Q , ARNHOLD J , GRASSBERGER P . Learning driver-response relationships from synchronization patterns[J]. Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics, 2000, 61(5 Pt A):5142-5148.

[70] ANDRZEJAK R G , KRASKOV A , STOGBAUER , et al. Bivariate surrogate techniques: Necessity, strengths, and caveats[J]. Physical Review E Statistical Nonlinear & Soft Matter Physics, 2003, 68(6):066202.

[71] CHICHARRO D , ANDRZEJAK R G . Reliable detection of directional couplings using rank statistics[J]. Physical Review E Statal Nonlinear & Soft Matter Physics, 2009, 80(2):026217.

[72] MAO X , SHANG P . Transfer entropy between multivariate time series[J]. Communications in Nonlinear Science and Numerical Simulation, 2016, 47:338-347.

[73] KRUG D , OSTERHAGE H , ELGER C E , et al. Estimating nonlinear interdependences in dynamical systems using cellular nonlinear networks.[J]. Physical Review E Statistical Nonlinear & Soft Matter Physics, 2007, 76(4):041916.

[74] ANNA KRAKOVSKÁ, JOZEF JAKUBÍK. Implementation of two causal methods based on predictions in reconstructed state spaces[J]. Physical Review E, 2020, 102:022203.

[75] CHEN S , JIN G , MA X . Satellite On-orbit Anomaly Detection Method Based on a Dynamic Threshold and Causality Pruning[J]. IEEE Access, 2021, PP(99):1-1.

[76] 张贵生. 数据驱动的金融时间序列预测模型研究[D].太原：山西大学,2016.

Zhang GS. Research on Data-Driven Financial Time Series Prediction Model [D]. Taiyuan: Shanxi University, 2016.

[77] CHIMOBI, OMOKE, PHILIP. The causal Relationship among Financial Development, Trade Openness and Economic Growth in Nigeria.[J]. International Journal of Economics & Finance, 2010，2(2):137-147.

[78] FIEDOR P . Causal Non-Linear Financial Networks[J]. Papers, arXiv:1407.5020v1[q-fin.ST] 2014.

[79] SEBRI M , BEN-SALHA O . On the causal dynamics between economic growth, renewable energy consumption, CO2 emissions and trade openness: Fresh evidence from BRICS countries[J]. Renewable & Sustainable Energy Reviews, 2014, 39:14-23.

[80] JUDGE G . Adaptive Intelligent Behavior and Causal Entropy Maximization as a Basis for Microeconomic Information Recovery[J]. SSRN Electronic Journal, 2014. Available at SSRN: http://dx.doi.org/10.2139/ssrn.2463323.

[81] LIU A , CHEN J , YANG S Y , et al. The Flow of Information in Trading: An Entropy Approach to Market Regimes[J]. Entropy, 2020, 22(9):1064.

[82] Sarkar M , Leong T Y . Characterization of medical time series using fuzzy similarity-based fractal dimensions[J]. Artificial Intelligence in Medicine, 2003, 27(2):201-222.

[83] WALTER L , FRANCISCO T , REDELICO F O , et al. Analysis of ischaemic crisis using the informational causal entropy-complexity plane[J]. Chaos, 2018, 28(7):075518-.

[84] STEPHAN K E , PENNY W D , MORAN R J , et al. Ten simple rules for dynamic causal modeling[J]. Neuroimage, 2010, 49( 4):3099-3109.

[85] LEE J , NEMATI S , SILVA I , et al. Transfer Entropy Estimation and Directional Coupling Change Detection in Biomedical Time Series[J]. BioMedical Engineering OnLine, 2012, 11(1):19.

[86] FAES L , MARINAZZO D , MONTALTO A , et al. Lag-specific transfer entropy as a tool to assess cardiovascular and cardiorespiratory information transfer[J]. IEEE transactions on bio-medical engineering, 2014, 61(10):2556-68.

[87] VALENZA G , FAES L , CITI L , et al. Instantaneous transfer entropy for the study of cardio-respiratory dynamics[C]// Proceedings of Engineering in Medicine & Biology Society. IEEE, 2015:7885-7888.

[88] PEREZ-SUAY A, CAMPS-VALLS G. Causal inference in geoscience and remote sensing from observational. Data[J]. IEEE Transactions on Geoscience and Remote Sensing. 2018, 3: 1502–1513.

[89] SILVA F N , VEGA-OLIVEROS D A , YAN X , et al. Detecting climate teleconnections with Granger causality[J]. https://doi.org/10.48550/arXiv.2012.03848, 2020.

[90] CONTRERAS-REYES J E , CAROLA HERNÁNDEZ-SANTORO. Assessing Granger-Causality in the Southern Humboldt Current Ecosystem Using Cross-Spectral Methods[J]. Entropy, 2020, 22(10):1071.

[91] LI J , CONVERTINO M . Inferring Ecosystem Networks as Information Flows[J]. Scientific Reports, 2021, 11:7094.

[92] OH M , KIM S , LIM K , et al. Time series analysis of the Antarctic Circumpolar Wave via symbolic transfer entropy[J]. Physica A Statistical Mechanics & Its Applications, 2018, 499:233-240.

[93] 郑树泉, 覃海焕, 王倩. 工业大数据技术与架构[J]. 大数据, 2017, 3(4):14.

Zheng S Q, Qin H H, Wang Q. Industrial Big Data Technology and Architecture [J]. Big Data Research, 2017, 3(4):14.

[94] YU W , YANG F . Detection of Causality between Process Variables Based on Industrial Alarm Data Using Transfer Entropy[J]. Entropy, 2015, 17(8):5868-5887.

[95] SHI D , GUO Z , JOHANSSON K H , et al. Causality Countermeasures for Anomaly Detection in Cyber-Physical Systems[J]. IEEE Transactions on Automatic Control, 2017, 63(2):386-401.

[96] ZENG Z, JIN G,XU C, et al. Satellite Telemetry Data Anomaly Detection Using Causal Network and Feature-Attention-based LSTM [J], IEEE Transactions on Instrumentation and Measurement, 2022, Early Access, doi:10.1109/TIM.2022.3151930.

[97] YAO C Z , LIN J N , LIN Q W , et al. A study of causality structure and dynamics in industrial electricity consumption based on Granger network[J]. Physica A: Statistical Mechanics and its Applications, 2016, 462:297-320.

[98] YOSHIDA T , NAKADAI K . Active audio-visual integration for Voice Activity Detection based on a Causal Bayesian Network[C]// Proceedings of IEEE-RAS International Conference on Humanoid Robots. IEEE, 2013:370-375.

[99] TIAN C , ZHAO C , FAN H , et al. Causal network construction based on convergent cross mapping (CCM) for alarm system root cause tracing of nonlinear industrial process[J]. IFAC-Papers Online, 2020, 53(2):13619-13624.