通信学报 ›› 2023, Vol. 44 ›› Issue (11): 94-109.doi: 10.11959/j.issn.1000-436x.2023209
• 专题:复杂环境下分布式边缘智能 • 上一篇
王冬1, 秦倩倩1, 郭开天1, 刘容轲1, 颜伟鹏1, 任一支1, 罗清彩2, 申延召3
修回日期:
2023-10-18
出版日期:
2023-11-01
发布日期:
2023-11-01
作者简介:
王冬(1990− ),女,山东泰安人,博士,杭州电子科技大学讲师,主要研究方向为人工智能安全、隐私计算等基金资助:
Dong WANG1, Qianqian QIN1, Kaitian GUO1, Rongke LIU1, Weipeng YAN1, Yizhi REN1, Qingcai LUO2, Yanzhao SHEN3
Revised:
2023-10-18
Online:
2023-11-01
Published:
2023-11-01
Supported by:
摘要:
联邦学习作为一种分布式机器学习技术可以解决数据孤岛问题,但机器学习模型会无意识地记忆训练数据,导致参与方上传的模型参数与全局模型会遭受各种隐私攻击。针对隐私攻击中的模型逆向攻击,对现有的攻击方法进行了系统总结。首先,概括并详细分析了模型逆向攻击的理论框架;其次,从威胁模型的角度对现有的攻击方法进行总结分析与比较;再次,总结与比较了不同技术类型的防御策略;最后,对现有模型逆向攻击常用的评估标准及数据集进行汇总,并对模型逆向攻击现有的主要挑战以及未来研究方向进行总结。
中图分类号:
王冬, 秦倩倩, 郭开天, 刘容轲, 颜伟鹏, 任一支, 罗清彩, 申延召. 联邦学习中的模型逆向攻防研究综述[J]. 通信学报, 2023, 44(11): 94-109.
Dong WANG, Qianqian QIN, Kaitian GUO, Rongke LIU, Weipeng YAN, Yizhi REN, Qingcai LUO, Yanzhao SHEN. Survey on model inversion attack and defense in federated learning[J]. Journal on Communications, 2023, 44(11): 94-109.
表2
模型逆向攻击的常用数据集"
数据类别 | 数据任务 | 数据集名称 | 样本个数 | 类别个数 | 特征维度 | 参考文献 |
图像 | 分类 | MNIST[ | 70 000 | 10 | 28×28×1 | 文献[15-16,19,21,32-33,37,39] |
Fashion-MNIST[ | 70 000 | 10 | 28×28×1 | 文献[ | ||
CIFAR-10[ | 60 000 | 10 | 32×32×3 | 文献[16,18-19,21,37-39] | ||
CelebA[ | 202 599 | 10 177 | 218×178×3 | 文献[15-16,19-22,25,33-34,37] | ||
FaceScrub[ | 100 000 | 530 | — | 文献[16,18,20-22,34,37-38] | ||
ChestX-Ray8[ | 108 948 | 32 717 | 1 024×1 024×1 | 文献[15-16,31] | ||
Stanford dogs[ | 20 580 | 120 | — | 文献[ | ||
AT&T Face[ | 400 | 40 | 92×112×1 | 文献[ | ||
Pubfig83[ | 13 600 | 83 | — | 文献[ | ||
VGGFace2[ | 3 310 000 | 9 131 | — | 文献[ | ||
生成 | Flickr-Faces-HQ[ | 70 000 | — | 1 024×1 024×3 | 文献[16,18-20,34,36] | |
MetFaces[ | 1 336 | — | 1 024×1 024×3 | 文献[ | ||
Animal Faces-HQ Dogs[ | 15 000 | — | 512×512×3 | 文献[ | ||
结构化 | 分类 | IWPC[ | 5 700 | — | — | 文献[ |
表3
联邦学习中模型逆向攻击和防御的主要挑战和未来研究方向"
研究内容 | 主要挑战 | 未来研究方向 |
模型逆向攻击 | 相同的模型逆向攻击方法无法针对不同领域的目标模型展开攻击 | 研究并探索具有普适性、适应性的跨领域模型逆向攻击方法 |
白盒模型逆向攻击方法训练攻击模型耗时较长,且无法针对同领域但不同结构和规模的目标模型进行迁移 | 研究可迁移的方法训练白盒模型逆向攻击的攻击模型 | |
黑盒模型逆向攻击方法在一些情况下的准确率较差,且无法在有限访问次数下进行有效的攻击 | 研究黑盒模型逆向攻击的框架,提出新的潜在向量优化方法,提升准确率,减少访问次数 | |
模型逆向攻击防御 | 基于深度学习训练技术的防御策略无法运用到各参与方的本地训练中 | 研究和设计一种普适性更强的本地模型训练策略 |
已有研究忽视防御策略的设置对于可用性和隐私性的影响,如差分隐私中隐私预算、模型参数大小 | 研究和设计一种隐私性与可用性均衡的防御策略,并探讨不同设置的影响 |
[54] | MA Y , YU D , WU T ,et al. PaddleFL:an open-source deep learning platform from industrial practice[J]. Frontiers of Data and Domputing, 2019,1(1): 105-115. |
[55] | 隐私计算联盟,中国信息通信研究院云计算与大数据研究院. 隐私计算应用研究报告[R]. 2022. |
Privacy Computing Alliance,Cloud Computing and Big Data Research Institute of China Academy of Information and Communications Technology. Privacy computing application research report[R]. 2022. | |
[56] | LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324. |
[57] | XIAO H , RASUL K , VOLLGRAF R . Fashion-MNIST:a novel image dataset for benchmarking machine learning algorithms[J]. arXiv Preprint,arXiv:1708.07747, 2017. |
[58] | KRIZHEVSKY A . Learning multiple layers of features from tiny images[D]. Tront:University of Tront, 2009. |
[59] | LIU Z W , LUO P , WANG X G ,et al. Deep learning face attributes in the wild[C]// Proceedings of IEEE International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2016: 3730-3738. |
[60] | NG H W , WINKLER S . A data-driven approach to cleaning large face datasets[C]// Proceedings of IEEE International Conference on Image Processing (ICIP). Piscataway:IEEE Press, 2015: 343-347. |
[61] | WANG X S , PENG Y F , LU L ,et al. ChestX-Ray8:hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 3462-3471. |
[62] | KHOSLA A , JAYADEVAPRAKASH N , YAO B ,et al. Novel dataset for fine-grained image categorization:Stanford dogs[C]// Workshop on Fine-grained Visual Categorization (FGVC). Piscataway:IEEE Press, 2011: 1-2. |
[63] | ADDLESEE M , CURWEN R , HODGES S ,et al. Implementing a sentient computing system[J]. Computer, 2001,34(8): 50-56. |
[1] | HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016: 770-778. |
[2] | WANG S , KANG B , MA J L ,et al. A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19)[J]. European Radiology, 2021,31(8): 6096-6104. |
[3] | 杨强 . AI与数据隐私保护:联邦学习的破解之道[J]. 信息安全研究, 2019,5(11): 961-965. |
YANG Q . AI and data privacy protection:the way to federated learning[J]. Journal of Information Security Research, 2019,5(11): 961-965. | |
[4] | MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[J]. arXiv Preprint,arXiv:1602.05629, 2016. |
[5] | SONG C Z , RISTENPART T , SHMATIKOV V . Machine learning models that remember too much[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2017: 587-601. |
[6] | 牛俊, 马骁骥, 陈颖 ,等. 机器学习中成员推理攻击和防御研究综述[J]. 信息安全学报, 2022,7(6): 1-30. |
NIU J , MA X J , CHEN Y ,et al. A survey on membership inference attacks and defenses in Machine Learning[J]. Journal of Cyber Security, 2022,7(6): 1-30. | |
[7] | SUN L C , QIAN J W , CHEN X . LDP-FL:practical private aggregation in federated learning with local differential privacy[C]// Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. California:International Joint Conferences on Artificial Intelligence Organization, 2021: 1571-1578. |
[8] | PAPERNOT N , ABADI M , ERLINGSSON ú , ,et al. Semi-supervised knowledge transfer for deep learning from private training data[J]. arXiv Preprint,arXiv:1610.05755, 2016. |
[9] | TRAMèR F , ZHANG F , JUELS A ,et al. Stealing machine learning models via prediction APIs[C]// Proceedings of the 25th USENIX Conference on Security Symposium. Berkeley:USENIX Association, 2016: 601-618. |
[10] | SHOKRI R , STRONATI M , SONG C Z ,et al. Membership inference attacks against machine learning models[C]// Proceedings of 2017 IEEE Symposium on Security and Privacy (SP). Piscataway:IEEE Press, 2017: 3-18. |
[11] | 刘艺璇, 陈红, 刘宇涵 ,等. 联邦学习中的隐私保护技术[J]. 软件学报, 2022,33(3): 1057-1092. |
LIU Y X , CHEN H , LIU Y H ,et al. Privacy-preserving techniques in federated learning[J]. Journal of Software, 2022,33(3): 1057-1092. | |
[12] | 陈明鑫, 张钧波, 李天瑞 . 联邦学习攻防研究综述[J]. 计算机科学, 2022,49(7): 310-323. |
CHEN M X , ZHANG J B , LI T R . Survey on attacks and defenses in federated learning[J]. Computer Science, 2022,49(7): 310-323. | |
[13] | FREDRIKSON M , LANTZ E , JHA S ,et al. Privacy in pharmacogenetics:an end-to-end case study of personalized warfarin dosing[C]// Proceedings of the USENIX Security Symposium. Berkeley:USENIX Association, 2014: 17-32. |
[14] | FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures[C]// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2015: 1322-1333. |
[15] | ZHANG Y H , JIA R X , PEI H Z ,et al. The secret revealer:generative model-inversion attacks against deep neural networks[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 250-258. |
[16] | CHEN S , KAHLA M , JIA R X ,et al. Knowledge-enriched distributional model inversion attacks[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2022: 16158-16167. |
[17] | GOODFELLOW I , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial networks[J]. Communications of the ACM, 2020,63(11): 139-144. |
[18] | STRUPPEK L , HINTERSDORF D , CORREIA A D A ,et al. Plug &play attacks:towards robust and flexible model inversion attacks[J]. arXiv Preprint,arXiv:2201.12179, 2022. |
[19] | NGUYEN N B , CHANDRASEGARAN K , ABDOLLAHZADEH M ,et al. Re-thinking model inversion attacks against deep neural networks[C]// Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2023: 16384-16393. |
[20] | YUAN X , CHEN K , ZHANG J ,et al. Pseudo label-guided model inversion attack via conditional generative adversarial network[J]. arXiv Preprint,arXiv:2302.09814, 2023. |
[21] | YANG Z Q , ZHANG J Y , CHANG E C ,et al. Neural network inversion in adversarial setting via background knowledge alignment[C]// Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2019: 225-240. |
[22] | KAHLA M , CHEN S , JUST H A ,et al. Label-only model inversion attacks via boundary repulsion[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2022: 15025-15033. |
[23] | XU R H , BARACALDO N , ZHOU Y ,et al. HybridAlpha:an efficient approach for privacy-preserving federated learning[C]// Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. New York:ACM Press, 2019: 13-23. |
[24] | CHENG K W , FAN T , JIN Y L ,et al. SecureBoost:a lossless federated learning framework[J]. IEEE Intelligent Systems, 2021,36(6): 87-98. |
[25] | ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2016: 308-318. |
[26] | WANG T H , ZHANG Y H , JIA R X . Improving robustness to model inversion attacks via mutual information regularization[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(13): 11666-11673. |
[27] | WEN J , YIU S M , HUI L C K . Defending against model inversion attack by adversarial examples[C]// Proceedings of 2021 IEEE International Conference on Cyber Security and Resilience (CSR). Piscataway:IEEE Press, 2021: 551-556. |
[28] | PENG X , LIU F , ZHANG J F ,et al. Bilateral dependency optimization:defending against model-inversion attacks[C]// Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2022: 1358-1367. |
[29] | GONG X L , WANG Z Y , CHEN Y J ,et al. NetGuard:protecting commercial Web APIs from model inversion attacks using GAN-generated fake samples[C]// Proceedings of the ACM Web Conference 2023. New York:ACM Press, 2023: 2045-2053. |
[30] | RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv Preprint,arXiv:1511.06434, 2015. |
[31] | WANG K C , FU Y , LI K ,et al. Variational model inversion attacks[J]. Advances in Neural Information Processing Systems, 2021,34: 9706-9719. |
[32] | MEHNAZ S , DIBBO S V , DE-VITI R , ,et al. Are your sensitive attributes private? novel model inversion attribute inference attacks on classification models[J]. arXiv Preprint,arXiv:2201.09370, 2022. |
[33] | DIONYSIOU A , VASSILIADES V , ATHANASOPOULOS E . Exploring model inversion attacks in the black-box setting[J]. Proceedings on Privacy Enhancing Technologies, 2023(1): 190-206. |
[34] | HAN G , CHOI J , LEE H ,et al. Reinforcement learning-based black-box model inversion attacks[C]// Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2023: 20504-20513. |
[35] | YOSHIMURA S , NAKAMURA K , NITTA N ,et al. Model inversion attack against a face recognition system in a black-box setting[C]// Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). Piscataway:IEEE Press, 2022: 1800-1807. |
[36] | AN S W , TAO G H , XU Q L ,et al. MIRROR:model inversion for deep learning network with high fidelity[C]// Proceedings of 2022 Network and Distributed System Security Symposium. Reston:Internet Society, 2022: 1-18. |
[37] | ZHU T Q , YE D Y , ZHOU S ,et al. Label-only model inversion attacks:attack with the least information[J]. IEEE Transactions on Information Forensics and Security, 2023,18: 991-1005. |
[38] | YIN Y P , ZHANG X L , ZHANG H L ,et al. Ginver:generative model inversion attacks against collaborative inference[C]// Proceedings of the ACM Web Conference. New York:ACM Press, 2023: 2122-2131. |
[39] | HE Z C , ZHANG T W , LEE R B . Model inversion attacks against collaborative inference[C]// Proceedings of the 35th Annual Computer Security Applications Conference. New York:ACM Press, 2019: 148-162. |
[40] | MIYATO T , KATAOKA T , KOYAMA M ,et al. Spectral normalization for generative adversarial networks[J]. arXiv Preprint,arXiv:1802.05957, 2018. |
[41] | KARRAS T , LAINE S , AILA T M . A style-based generator architecture for generative adversarial networks[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 4396-4405. |
[42] | BLEI D M , KUCUKELBIR A , MCAULIFFE J D . Variational inference:a review for statisticians[J]. Journal of the American Statistical Association, 2017,112(518): 859-877. |
[43] | CARLINI N , WAGNER D . Towards evaluating the robustness of neural networks[C]// Proceedings of 2017 IEEE Symposium on Security and Privacy (SP). Piscataway:IEEE Press, 2017: 39-57. |
[44] | SRIRAMANAN G , ADDEPALLI S , BABURAJ A ,et al. Guided adversarial attack for evaluating and enhancing adversarial defenses[J]. arXiv Preprint,arXiv:2011.14969, 2020. |
[45] | CHOQUETTE-CHOO C A , DULLERUD N , DZIEDZIC A ,et al. CaPC learning:confidential and private collaborative learning[J]. arXiv Preprint,arXiv:2102.05188, 2021. |
[46] | LI Z , ZHANG Y . Membership leakage in label-only exposures[C]// Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2021: 880-895. |
[47] | CHOQUETTE-CHOO C A , TRAMER F , CARLINI N ,et al. Label-only membership inference attacks[J]. arXiv Preprint,arXiv:2007.14321, 2020. |
[48] | BHANDARI D , MURTHY C A , PAL S K . Genetic algorithm with elitist model and its convergence[J]. International Journal of Pattern Recognition and Artificial Intelligence, 1996,10(6): 731-747. |
[49] | YANG Z , WANG L , YANG D ,et al. Purifier:defending data inference attacks via transforming confidence scores[J]. arXiv Preprint,arXiv:2212.00612, 2022. |
[50] | 微众银行AI项目组. 联邦学习白皮书 V1.0[R]. 2019. |
Project Team. Federated learning white paper V1.0[R]. 2019. | |
[64] | PINTO N , STONE Z , ZICKLER T ,et al. Scaling up biologically-inspired computer vision:a case study in unconstrained face recognition on facebook[C]// Proceedings of CVPR 2011 WORKSHOPS. Piscataway:IEEE Press, 2011: 35-42. |
[65] | CAO Q , SHEN L , XIE W D ,et al. VGGFace2:a dataset for recognising faces across pose and age[C]// Proceedings of 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). Piscataway:IEEE Press, 2018: 67-74. |
[66] | KARRAS T , AITTALA M , HELLSTEN J ,et al. Training generative adversarial networks with limited data[J]. arXiv Preprint,arXiv:2006.06676, 2020. |
[67] | CHOI Y , UH Y , YOO J ,et al. StarGAN v2:diverse image synthesis for multiple domains[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 8185-8194. |
[68] | The International Warfarin Pharmacogenetics Consortium. Estimation of the warfarin dose with clinical and pharmacogenetic data[J]. New England Journal of Medicine, 2009,360(8): 753-764. |
[69] | 贾轩, 白玉真, 马智华 . 隐私计算应用场景综述[J]. 信息通信技术与政策, 2022(5): 45-52. |
JIA X , BAI Y Z , MA Z H . Overview of privacy preserving computing application scenarios[J]. Information and Communications Technology and Policy, 2022(5): 45-52. | |
[51] | 微众银行AI项目组. 联邦学习开源平台 FATE[R]. 2019. |
WeBank AI Project Team. Federated learning open source platform FATE[R]. 2019. | |
[52] | ZILLER A , TRASK A , LOPARDO A ,et al. Pysyft:a library for easy federated learning[J]. Federated Learning Systems:Towards Next-Generation AI, 2021: 111-139. |
[53] | CHEN Y A , HUANG G , SHI J ,et al. Rosetta:a privacy-preserving framework based on TensorFlow[E]. 2020. |
[1] | 陈晓霖, 昝道广, 吴炳潮, 关贝, 王永吉. 面向纵向联邦学习的对抗样本生成算法[J]. 通信学报, 2023, 44(8): 1-13. |
[2] | 马卓, 金嘉玉, 杨易龙, 刘洋, 应作斌, 李腾, 张俊伟. 基于门限同态加密的自适应联邦学习安全聚合方案[J]. 通信学报, 2023, 44(7): 76-85. |
[3] | 马鑫迪, 李清华, 姜奇, 马卓, 高胜, 田有亮, 马建峰. 面向Non-IID数据的拜占庭鲁棒联邦学习[J]. 通信学报, 2023, 44(6): 138-153. |
[4] | 金彪, 李逸康, 姚志强, 陈瑜霖, 熊金波. GenFedRL:面向深度强化学习智能体的通用联邦强化学习框架[J]. 通信学报, 2023, 44(6): 183-197. |
[5] | 张佳乐, 朱诚诚, 孙小兵, 陈兵. 基于GAN的联邦学习成员推理攻击与防御方法[J]. 通信学报, 2023, 44(5): 193-205. |
[6] | 田有亮, 吴柿红, 李沓, 王林冬, 周骅. 基于激励机制的联邦学习优化算法[J]. 通信学报, 2023, 44(5): 169-180. |
[7] | 姜慧, 何天流, 刘敏, 孙胜, 王煜炜. 面向异构流式数据的高性能联邦持续学习算法[J]. 通信学报, 2023, 44(5): 123-136. |
[8] | 余晟兴, 陈泽凯, 陈钟, 刘西蒙. DAGUARD:联邦学习下的分布式后门攻击防御方案[J]. 通信学报, 2023, 44(5): 110-122. |
[9] | 李开菊, 许强, 王豪. 冗余数据去除的联邦学习高效通信方法[J]. 通信学报, 2023, 44(5): 79-93. |
[10] | 马千飘, 贾庆民, 刘建春, 徐宏力, 谢人超, 黄韬. 异构边缘计算环境下异步联邦学习的节点分组与分时调度策略[J]. 通信学报, 2023, 44(11): 79-93. |
[11] | 汤凌韬, 王迪, 刘盛云. 面向非独立同分布数据的联邦学习数据增强方案[J]. 通信学报, 2023, 44(1): 164-176. |
[12] | 余晟兴, 陈钟. 基于同态加密的高效安全联邦学习聚合框架[J]. 通信学报, 2023, 44(1): 14-28. |
[13] | 范绍帅, 吴剑波, 田辉. 面向能量受限工业物联网设备的联邦学习资源管理[J]. 通信学报, 2022, 43(8): 65-77. |
[14] | 莫梓嘉, 高志鹏, 杨杨, 林怡静, 孙山, 赵晨. 面向车联网数据隐私保护的高效分布式模型共享策略[J]. 通信学报, 2022, 43(4): 83-94. |
[15] | 康海燕, 冀源蕊. 基于本地化差分隐私的联邦学习方法研究[J]. 通信学报, 2022, 43(10): 94-105. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||
|