人工智能风险分析技术研究进展

doi:10.11959/j.issn.2096-0271.2020005

大数据 ›› 2020, Vol. 6 ›› Issue (1): 47-59.doi: 10.11959/j.issn.2096-0271.2020005

人工智能风险分析技术研究进展

陈群^1,²,陈肇强^1,²,侯博议^1,²,王丽娟^1,²,罗雨晨^1,²,李战怀^1,²

¹ 西北工业大学计算机学院，陕西西安 710129
² 西北工业大学大数据存储与管理工业和信息化部重点实验室，陕西西安 710129

出版日期:2020-01-15 发布日期:2020-02-21
作者简介:陈群（1976- ），男，博士，西北工业大学计算机学院教授、博士生导师，主要研究方向为人工智能风险分析等大数据分析技术|陈肇强（1988- ），男，西北工业大学计算机学院博士生，主要研究方向为人工智能风险分析等大数据分析技术|侯博议（1990- ），男，西北工业大学计算机学院博士生，主要研究方向为人工智能风险分析等大数据分析技术|王丽娟（1992- ），女，西北工业大学计算机学院硕士生，主要研究方向为人工智能风险分析等大数据分析技术|罗雨晨（1997- ），女，西北工业大学计算机学院硕士生，主要研究方向为人工智能风险分析等大数据分析技术|李战怀（1961- ），男，博士，西北工业大学计算机学院教授、博士生导师，大数据存储与管理工业和信息化部重点实验室主任，主要研究方向为大数据管理技术、海量信息存储系统等
基金资助:
国家重点研发计划基金资助项目(2018YFB1003400);国家自然科学基金资助项目(61732014);国家自然科学基金资助项目(61672432);陕西省自然科学基础研究计划基金资助项目(2018JM6086)

Research progress on risk analysis for artificial intelligence

Qun CHEN^1,²,Zhaoqiang CHEN^1,²,Boyi HOU^1,²,Lijuan WANG^1,²,Yuchen LUO^1,²,Zhanhuai LI^1,²

¹ School of Computer Science,Northwestern Polytechnical University,Xi’an 710129,China
² Key Laboratory of Big Data Storage and Management,Northwestern Polytechnical University,Ministry of Industry and Information Technology,Xi’an 710129,China

Online:2020-01-15 Published:2020-02-21
Supported by:
The National Key Research and Development Program of China(2018YFB1003400);The National Natural Science Foundation of China(61732014);The National Natural Science Foundation of China(61672432);The Project Supported by Natural Science Basic Research Plan in Shaanxi Province of China(2018JM6086)

摘要/Abstract

摘要：

目前基于深度学习模型的预测在真实场景中具有不确定性和不可解释性，给人工智能应用的落地带来了不可避免的风险。首先阐述了风险分析的必要性以及其需要具备的3个基本特征：可量化、可解释、可学习。接着，分析了风险分析的研究现状，并重点介绍了笔者最近提出的一个可量化、可解释和可学习的风险分析技术框架。最后，讨论风险分析的现有以及潜在的应用，并展望其未来的研究方向。

关键词: 人工智能, 风险分析, 不确定性, 可解释性

Abstract:

The predictions of the deep learning models are still uncertain and uninterpretable.As a result,their deployments bring unavoidable risk to business decision making.Firstly,the study on risk analysis was motivated,and the three desirable properties of risk analysis techniques were described:quantifiability,interpretability and learnability.Then the existing work on risk analysis was reviewed,and the newly proposed framework to enable quantifiable,interpretable and learnable risk analysis was introduced.Finally,the existing and potential applications of risk analysis,and its future research direction were discussed.

Key words: artificial intelligence, risk analysis, uncertainty, interpretability

中图分类号:

TP391

陈群, 陈肇强, 侯博议, 王丽娟, 罗雨晨, 李战怀. 人工智能风险分析技术研究进展[J]. 大数据, 2020, 6(1): 47-59.

Qun CHEN, Zhaoqiang CHEN, Boyi HOU, Lijuan WANG, Yuchen LUO, Zhanhuai LI. Research progress on risk analysis for artificial intelligence[J]. Big Data Research, 2020, 6(1): 47-59.

图/表 5

参考文献 31

[1]	齐治昌, 谭庆平, 宁洪 . 软件工程[M]. 北京: 高等教育出版社, 2001： 304-331.
	QI Z C , TAN Q P , NING H . Software engineering[M]. Beijing: Higher Education PressPress, 2001: 304-331.
[2]	AMODEI D , OLAH C , STEINHARDT J ,et al. Concrete problems in AI safety[J]. Computer Science, 2016,arXiv:1606.06565.
[3]	HENDRYCKS D , GIMPEL K . A baseline for detecting misclassified and outof-distribution examples in neural networks[C]// The 5th International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:s.n]. 2017: 1-21.
[4]	CORBETT-DAVIES S , GOEL S . The measure and mismeasure of fairness:a critical review of fair machine learning[J]. Computer Science, 2018,arXiv:1808.00023.
[5]	MORLEY J , FLORIDI L , KINSEY L ,et al. From what to how:an overview of AI ethics tools,methods and research to translate principles into practices[J]. Computer Science, 2019,arXiv:1905.06876.
[6]	JIANG H , KIM B , GUAN M ,et al. To trust or not to trust a classifier[C]// Advances in Neural Information Processing Systems,December 3,2018,Montreal,Canada.[S.l.:s.n]. 2018: 5541-5552.
[7]	KOHAVI R , . A study of cross-validation and bootstrap for accuracy estimation and model selection[C]// The 14th International Joint Conference on Artificial Intelligence,August 20-25,1995,Montreal,Canada.San Francisco:Morgan Kaufmann Publishers Inc. 1995: 1137-1143.
[8]	RIBEIRO M T , SINGH S , GUESTRIN C . Why should I trust you? explaining the predictions of any classifier[C]// The 22nd ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD),August 24-27,2016,San Francisco,USA. New York:ACM Press, 2016: 1135-1144.
[9]	GUIDOTTI R , MONREALE A , RUGGIERI S ,et al. A survey of methods for explaining black box models[J]. ACM Computing Surveys, 2019,51(5):93.
[10]	RIBEIRO M T , SINGH S , GUESTRIN C . Anchors:high-precision model-agnostic explanations[C]// The 32nd AAAI Conference on Artificial Intelligence,February 2-7,2018,New Orleans,USA.[S.l.:s.n]. 2018: 1527-1535.
[11]	BAEHRENS D , SCHROETER T , HARMELING S ,et al. How to explain individual classification decisions[J]. Journal of Machine Learning Research, 2010,11(11): 1803-1831.
[12]	KONONENKO I , STRUMBELJ E . An efficient explanation of individual classifications using game theory[J]. Journal of Machine Learning Research, 2010,11(1): 1-18.
[13]	AMERSHI S , CHICKERING M , DRUCKER S ,et al. Modeltracker:redesigning performance analysis tools for machine learning[C]// The 33rd Annual ACM Conference on Human Factors in Computing Systems,April 18-23,2015,Seoul,Korea. New York:ACM Press, 2015: 337-346.
[14]	TOLOMEI G , SILVESTRI F , HAINES A ,et al. Interpretable predictions of treebased ensembles via actionable feature tweaking[C]// The 23rd ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD),August 13-17,2017,Halifax,Canada. New York:ACM Press, 2017: 465-474.
[15]	KONONENKO I , . Semi-naive Bayesian classifier[C]// The 6th European Working Session on Learning,March 6-8,1991,Porto,Portugal. London:SpringerVerlag, 1991: 206-219.
[16]	PLATT J . Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods[J]. Advances in Large Margin Classifiers, 1999,10(3): 61-74.
[17]	HENDRYCKS D , MAZEIKA M , DIETTERICH T . Deep anomaly detection with outlier exposure[C]// The 7th International Conference on Learning Representations (ICLR),May 6-9,2019,New Orleans,USA.[S.l.:s.n]. 2019: 1-18.
[18]	OVADIA Y , FERTIG E , REN J ,et al. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift[J]. Computer Science, 2019,arXiv:1906.02530.
[19]	ZHANG P , WANG J L , FARHADI A ,et al. Predicting failures of vision systems[C]// The IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus,USA. Piscataway:IEEE Press, 2014: 3566-3573.
[20]	BANSAL A , FARHADI A , PARIKH D . Towards transparent systems:semantic characterization of failure modes[C]// The 13th European Conference on Computer Vision,September 6-12,2014,Zurich,Switzerland. Heidelberg:Springer, 2014: 366-381.
[21]	HOU B Y , CHEN Q , CHEN Z Q ,et al. r-HUMO:a risk-aware human-machine cooperation framework for entity resolution with quality guarantees[J]. IEEE Transactions on Knowledge and Data Engineering, 2018: 1-13.
[22]	CHEN Z Q , CHEN Q , HOU B Y ,et al. Improving machine-based entity resolution with limited human effort:a risk perspective[C]// The International Workshop on Real-Time Business Intelligence and Analytics,August 27,2018,Rio de Janeiro,Brazil.[S.l.:s.n. ], 2018: 1-5.
[23]	Chen Z Q , Chen Q , Hou B Y ,et al. Towards interpretable and learnable risk analysis for entity resolution[R]. 2019: 1-32.
[24]	VENTURA F , CERQUITELLI T . What’s in the box? explaining the black-box model through an evaluation of its interpretable features[J].,2019,arXiv:1908.04348. Computer Science, 2019:arXiv:1908.04348.
[25]	ANIS K , ZAKIA H , MOHAMED D ,et al. Detecting depression severity by interpretable representations of motion dynamics[C]// The 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG),May 15-19,2018,Xi’an,China. Piscataway:IEEE Press, 2018: 739-745.
[26]	LICON C , BOSC G , SABRI M ,et al. Chemical features mining provides new descriptive structure-odor relationships[J]. PLoS Computational Biology, 2019,15(4):e1006945.
[27]	MARKOWITZ H M , BLAY K A . 风险?收益分析:理性投资的理论与实践(第一卷)[M]. 唐亮,武微,译.北京: 机械工业出版社, 2016： 100-121
	MARKOWITZ H M , BLAY K A . Riskreturn analysis:the theory and practice of rational investing(Volume 1)[M]. Translated by TANG L,WU W. Beijing: China Machine PressPress, 2016: 100-121.
[28]	LI G L , WANG J N , ZHENG Y D ,et al. Crowdsourced data management:a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2016,28(9): 2296-2319.
[29]	CHEN Z Q , CHEN Q , FAN F F ,et al. Enabling quality control for entity resolution:a human and machine cooperation framework[C]// IEEE 34th International Conference on Data Engineering (ICDE),April 16-19,2018,Paris,France. Piscataway:IEEE Press, 2018: 1156-1167.
[30]	GISSIN D,SHALEV-SHWARTZ S . Discriminative active learning[J]. 2019,arXiv:1907.06347.
[31]	GUO C , PLEISS G , SUN Y ,et al. On calibration of modern neural networks[C]// The 34th International Conference on Machine Learning,August 6-11,2017,Sydney,Australia.[S.l.:s.n]. 2017: 1321-1330.

人工智能风险分析技术研究进展

Research progress on risk analysis for artificial intelligence

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 31

相关文章 15

Metrics

推荐阅读 0

[1]	王皓, 潘昱杉, 潘毅. 生成式人工智能大模型赋能的元宇宙生命体：前瞻和挑战[J]. 大数据, 2023, 9(3): 85-96.
[2]	李婧文, 李雅文. 算法应用风险与治理研究[J]. 大数据, 2023, 9(3): 140-149.
[3]	刘业政, 宗兰芳, 金斗, 袁昆. 数据要素流通使用的安全风险分析及应对策略[J]. 大数据, 2023, 9(2): 79-98.
[4]	沈阳, 余梦珑. 元宇宙与大数据：时空智能中的数据洞察与价值连接[J]. 大数据, 2023, 9(1): 103-110.
[5]	赵智韬, 赵理君, 张正, 唐娉. 基于容器云技术的典型遥感智能解译算法集成[J]. 大数据, 2022, 8(2): 58-74.
[6]	王丽会, 秦永彬. 深度学习在医学影像中的研究进展及发展趋势[J]. 大数据, 2020, 6(6): 83-104.
[7]	夏正勋,杨一帆,罗圣美,赵大超,张燕,唐剑飞. 生成技术在人工智能平台中的应用探索[J]. 大数据, 2020, 6(6): 0-.
[8]	于璠. 新一代深度学习框架研究[J]. 大数据, 2020, 6(4): 69-80.
[9]	缐珊珊. 美俄人工智能军事应用发展分析[J]. 大数据, 2020, 6(4): 125-132.
[10]	孟小峰, 王雷霞, 刘俊旭. 人工智能时代的数据隐私、垄断与公平[J]. 大数据, 2020, 6(1): 35-46.
[11]	李肯立, 刘楚波. 边缘智能：现状和展望[J]. 大数据, 2019, 5(3): 69-75.
[12]	周知, 于帅, 陈旭. 边缘智能：边缘计算与人工智能融合的新范式[J]. 大数据, 2019, 5(2): 53-63.
[13]	韩冬, 李其花, 蔡巍, 夏雨薇, 宁佳, 黄峰. 人工智能在医学影像中的研究与应用[J]. 大数据, 2019, 5(1): 39-67.
[14]	卢亿雷. 混合异构营销数据平台实践[J]. 大数据, 2018, 4(3): 104-110.
[15]	林华生, 钱岭, 周莹, 杨希, 李婧. 垃圾短信大数据自动识别技术[J]. 大数据, 2018, 4(2): 115-120.