XAI架构探索与实践

doi:10.11959/j.issn.2096-0271.2024013

大数据 ›› 2024, Vol. 10 ›› Issue (1): 86-109.doi: 10.11959/j.issn.2096-0271.2024013

• 应用 • 上一篇

XAI架构探索与实践

夏正勋¹, 唐剑飞¹, 杨一帆¹, 罗圣美², 张燕¹, 谭锋镭¹, 谭圣儒¹

¹ 星环信息科技（上海）股份有限公司，上海 200233
² 中孚信息股份有限公司，江苏南京 211899

出版日期:2024-01-01 发布日期:2024-01-01
作者简介:夏正勋（1979- ），男，星环信息科技（上海）股份有限公司高级研究员，主要研究方向为人工智能、大数据、数据库、流媒体处理技术。
唐剑飞（1986- ），男，星环信息科技（上海）股份有限公司大数据技术标准研究员，主要研究方向为大数据、数据库、图计算。
杨一帆（1986- ），男，博士，星环信息科技（上海）有限公司产品总监、首席科学家，主要研究方向为统计（统计计算、生存分析、时间序列和生物信息）、图计算、强化学习。
罗圣美（1971- ），男，博士，中孚信息股份有限公司副总裁，主要研究方向为大数据、数据安全、人工智能。
张燕（1985- ），女，星环信息科技（上海）股份有限公司大数据技术研究员，主要研究方向为大数据、人工智能等。
谭锋镭（1990- ），男，就职于星环信息科技（上海）股份有限公司，主要研究方向为AI平台、分布式系统研发。
谭圣儒（1995- ），男，星环信息科技（上海）股份有限公司人工智能产品经理，主要研究方向为机器学习、模型管理、模型运维监控、模型可解释性等。

Exploration and practice of XAI architecture

Zhengxun XIA¹, Jianfei TANG¹, Yifan YANG¹, Shengmei LUO², Yan ZHANG¹, Fenglei TAN¹, Shengru TAN¹

¹ Transwarp Information Technology (Shanghai) Co., Ltd., Shanghai 200233, China
² Zhongfu Information Inc., Nanjing 211899, China

Online:2024-01-01 Published:2024-01-01

摘要/Abstract

摘要：

可解释AI（explainable AI，XAI）是可信AI技术的重要组成。当前，业界对XAI的技术点展开了深入的研究，但在工程化实施方面尚缺少系统性研究。提出了一种通用的XAI技术架构，从原子解释生成、核心能力增强、业务组件嵌入、可信解释应用4个方面入手，设计了XAI基础能力层、XAI核心能力层、XAI业务组件层、XAI应用层4个层级，通过各技术层之间的分工协作，XAI工程化的落地实施得到了全流程保障。基于该XAI架构，可以灵活地引入新的技术模块，支撑XAI的产业化应用，为XAI在行业中的推广提供了一定的参考。

关键词: 可解释AI, 可信AI, XAI架构

Abstract:

XAI(explainable AI) is an important component of trusted AI.In-depth research on the technology points of XAI has been carried out in the current industry, but systematic research on engineering implementation is lacking.This paper proposed a general XAI technical architecture, which started from the follow four aspects: atomic interpretation generation, core competence enhancement, business component embedding and trusted interpretation application.We designed four levels:XAI foundation layer, XAI core competence layer, XAI business component layer and XAI application layer.Through the division of labor and cooperation among the technical layers, the implementation of XAI engineering was guaranteed throughout the whole process.Based on the XAI architecture presented in this paper, new technical modules can be introduced flexibly to support the industrialization application of XAI, providing certain reference for the promotion of XAI in the industry.

Key words: explainable AI, trusted AI, XAI architecture

中图分类号:

F08
F062.5

夏正勋, 唐剑飞, 杨一帆, 罗圣美, 张燕, 谭锋镭, 谭圣儒. XAI架构探索与实践[J]. 大数据, 2024, 10(1): 86-109.

Zhengxun XIA, Jianfei TANG, Yifan YANG, Shengmei LUO, Yan ZHANG, Fenglei TAN, Shengru TAN. Exploration and practice of XAI architecture[J]. Big Data Research, 2024, 10(1): 86-109.

图/表 7

图1

图2

图3

图4

图5

图6

图7

参考文献 44

[1]	叶津汶 . 大数据、人工智能在金融领域应用研究[J]. 理财, 2022(8): 13-15.
	YE J W . Research on the application of big data and artificial intelligence in the financial field[J]. Finance, 2022(8): 13-15.
[2]	谢芬 . 人工智能技术及其在智能机器人领域的应用[C]// 2018年智慧教育与人工智能发展学术会议论文集. 香港:香港新世纪文化出版社, 2018: 21-23.
	XIE F . Artificial intelligence technology and its application in intelligent robot field[C]// Proceedings of the 2018 Intelligent Education and Artificial Intelligence Development Conference. Hong Kong:Hong Kong New Century Culture Publishing House, 2018: 21-23.
[3]	杨晓光, 马成元, 王一喆 ,等. 交通人工智能及其发展综述研究[J]. 人工智能, 2022(4): 18-29.
	YANG X G , MA C Y , WANG Y Z ,et al. Research on artificial intelligence in transportation and its development[J]. Artificial Intelligence, 2022(4): 18-29.
[4]	高奇琦, 吕俊延 . 智能医疗:人工智能时代对公共卫生的机遇与挑战[J]. 电子政务, 2017(11): 11-19.
	GAO Q Q , LYU J Y . Intelligent healthcare:opportunities and challenges to public health in the era of artificial Intelligence[J]. E-government, 2017(11): 11-19.
[5]	刘慈欣 . 三体[J]. 意林, 2019(12): 67.
	LIU C X . The three-body problem[J]. Yilin, 2019(12): 67.
[6]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence entering the post deep learning era[J]. Journal of Intelligent Science and Technology, 2019,1(1): 4-6.
[7]	夏正勋, 唐剑飞, 罗圣美 ,等. 可信AI治理框架探索与实践[J]. 大数据, 2022,8(4): 145-164.
	XIA Z X , TANG J F , LUO S M ,et al. Exploration and practice of trusted AI governance framework[J]. Big Data Research, 2022,8(4): 145-164.
[8]	RIBEIRO M T , SINGH S , GUESTRIN C . “Why should I trust you?”:explaining the predictions of any classifier[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2016: 1135-1144.
[9]	ZHOU B L , KHOSLA A , LAPEDRIZA A ,et al. Learning deep features for discriminative localization[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 2921-2929.
[10]	AKULA A , WANG S , ZHU S C . CoCoX:generating conceptual and counterfactual explanations via fault-lines[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(3): 2594-2601.
[11]	HSIEH C , MOREIRA C , CHUN O Y . DiCE4EL:interpreting process predictions using a milestone-aware counterfactual approach[C]// Proceedings of 2021 3rd International Conference on Process Mining (ICPM). Piscataway:IEEE Press, 2021: 88-95.
[12]	RUBIN D B . Causal inference using potential outcomes[J]. Journal of the American Statistical Association, 2005,100(469): 322-331.
[13]	JAFTA G , DE WAAL A , DERKS I ,et al. Evaluation of XAI as an enabler for fairness,accountability and transparency[C]// Proceedings of the Second Southern African Conference for Artificial Intelligence Research. Cham:Springer, 2022.
[14]	PRADHAN R , ZHU J , GLAVIC B ,et al. Interpretable data-based explanations for fairness debugging[EB]. arXiv preprint, 2021,arXiv:2112.09745.
[15]	DU M N , LIU N H , HU X . Techniques for interpretable machine learning[J]. Communications of the ACM, 2020,63(1): 68-77.
[16]	曾春艳, 严康, 王志锋 ,等. 深度学习模型可解释性研究综述[J]. 计算机工程与应用, 2021,57(8): 1-9.
	ZENG C Y , YAN K , WANG Z F ,et al. Survey of interpretability research on deep learning models[J]. Computer Engineering and Applications, 2021,57(8): 1-9.
[17]	PáEZ A . The pragmatic turn in explainable artificial intelligence (XAI)[J]. Minds and Machines, 2019,29(3): 441-459.
[18]	MILLER T . Explanation in artificial intelligence:insights from the social sciences[J]. Artificial Intelligence, 2019,267: 1-38.
[19]	KULESZA T , STUMPF S , WONG W K ,et al. Why-oriented end-user debugging of naive Bayes text classification[J]. ACM Transactions on Interactive Intelligent Systems. 2011,1(1): 1-31.
[20]	HSIAO J H W , NGAI H H T , QIU L ,et al. Roadmap of designing cognitive metrics for explainable artificial intelligence (XAI)[EB]. arXiv preprint, 2021,arXiv:2108.01737.
[21]	SZCZEPA?SKI M , CHORA? M , PAWLICKI M ,et al. The methods and approaches of explainable artificial intelligence[C]// Proceedings of the International Conference on Computational Science. Cham:Springer, 2021: 3-17.
[22]	SPINNER T , SCHLEGEL U , SCH?FER H , ,et al. ExplAIner:a visual analytics framework for interactive and explainable machine learning[J]. IEEE Transactions on Visualization and Computer Graphics, 2020,26(1): 1064-1074.
[23]	KWON B C , CHOI M J , KIM J T ,et al. RetainVis:visual analytics with interpretable and interactive recurrent neural networks on electronic medical records[J]. IEEE Transactions on Visualization and Computer Graphics, 2019,25(1): 299-309.
[24]	OHANA J J , OHANA S , BENHAMOU E ,et al. Explainable AI (XAI) models applied to the multi-agent environment of financial markets[C]// Proceedings of the Explainable and Transparent AI and Multi-Agent Systems. Cham:Springer, 2021: 189-207.
[25]	PARK S , YANG J S . Interpretable deep learning LSTM model for intelligent economic decision-making[J]. KnowledgeBased Systems, 2022,248:108907.
[26]	周健民, 沈仁芳 . 土壤学大辞典[M]. 北京: 科学出版社, 2013.
	ZHOU J M , SHEN R F . Dictionary of soil science[M]. Beijing: Science Press, 2013.
[27]	SELVARAJU R R , COGSWELL M , DAS A ,et al. Grad-CAM:visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020,128(2): 336-359.
[28]	BACH S , BINDER A , MONTAVON G ,et al. On pixel-wise explanations for nonlinear classifier decisions by layer-wise relevance propagation[J]. PLoS One, 2015,10(7): e0130140.
[29]	SHRIKUMAR A , GREENSIDE P , KUNDAJE A . Learning important features through propagating activation differences[C]// Proceedings of the 34th International Conference on Machine Learning. New York:ACM, 2017: 3145-3153.
[30]	SPRINGENBERG J T , DOSOVITSKIY A , BROX T ,et al. Striving for simplicity:the all convolutional net[EB]. arXiv preprint, 2014,arXiv:1412.6806.
[31]	LUNDBERG S , LEE S I . A unified approach to interpreting model predictions[EB]. arXiv preprint, 2017
[32]	孔祥维, 唐鑫泽, 王子明 . 人工智能决策可解释性的研究综述[J]. 系统工程理论与实践, 2021,41(2): 524-536.
	KONG X W , TANG X Z , WANG Z M . A survey of explainable artificial intelligence decision[J]. Systems Engineering-Theory＆ Practice, 2021,41(2): 524-536.
[33]	MILLER G . WordNet:a lexical database for English[J]. Commun ACM, 1995,38: 39-41.
[34]	SCHOONDERWOERD T A J , JORRITSMA W , NEERINCX M A ,et al. Human-centered XAI:developing design patterns for explanations of clinical decision support systems[J]. International Journal of Human-Computer Studies, 2021,154:102684.
[35]	LIAO Q V , VARSHNEY K R . Humancentered explainable AI (XAI):from algorithms to user experiences[EB]. arXiv preprint, 2022,arXiv:2110.10790.
[36]	BREIMAN L . Statistical modeling:the two cultures (with comments and a rejoinder by the author)[J]. Statistical Science, 2001,16(3): 199-231.
[37]	OUYANG L , WU J , XU J ,et al. Training language models to follow instructions with human feedback[EB]. arXiv preprint, 2022.
[38]	BHAT S , JIANG J Q , POOLADZANDI O B ,et al. De-biasing generative models using counterfactual methods[EB]. arXiv preprint, 2022.
[39]	JOHANSSON F , SHALIT U , SONTAG D . Learning representations for counterfactual inference[C]// Proceedings of the International Conference on Machine Learning.[S.l.:s.n.], 2016: 3020-3029.
[40]	PEARL J , MACKENZIE D . The book of why:the new science of cause and effect[M]. New York: AAAS, 2018: 855-855.
[41]	王萍 . A/B测试方法的教育应用研究[J]. 电化教育研究, 2015,36(8): 58-66.
	WANG P . Research on the educational application of A/B testing method[J]. e-Education research, 2015,36(8): 58-66.
[42]	YANG B , SAILER A , JAIN S ,et al. Service discovery based blue-green deployment technique in cloud native environments[C]// Proceedings of 2018 IEEE International Conference on Services Computing (SCC). Piscataway:IEEE Press, 2018: 185-192.
[43]	KIM E , LEE J , SHIN H ,et al. Championchallenger analysis for credit card fraud detection:hybrid ensemble and deep learning[J]. Expert Systems With Applications, 2019,128: 214-224.
[44]	刘曾荣, 李挺 . 复杂系统理论剖析[J]. 自然杂志, 2004,26(3): 149-151.
	LIU Z R , LI T . Analysis of complex system[J]. Ziran Zazhi, 2004,26(3): 149-151.

XAI架构探索与实践

Exploration and practice of XAI architecture

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 44

相关文章 1

Metrics

推荐阅读 0