网络与信息安全学报 ›› 2021, Vol. 7 ›› Issue (5): 77-92.doi: 10.11959/j.issn.2096-109x.2021056
所属专题: 联邦学习
周传鑫, 孙奕, 汪德刚, 葛桦玮
修回日期:
2020-10-10
出版日期:
2021-10-15
发布日期:
2021-10-01
作者简介:
周传鑫(1997− ),男,安徽蚌埠人,信息工程大学硕士生,主要研究方向为数据安全交换、机器学习和隐私保护基金资助:
Chuanxin ZHOU, Yi SUN, Degang WANG, Huawei GE
Revised:
2020-10-10
Online:
2021-10-15
Published:
2021-10-01
Supported by:
摘要:
联邦学习由于能够在多方数据源聚合的场景下协同训练全局最优模型,近年来迅速成为安全机器学习领域的研究热点。首先,归纳了联邦学习定义、算法原理和分类;接着,深入分析了其面临的主要威胁与挑战;然后,重点对通信效率、隐私安全、信任与激励机制3个方向的典型研究方案对比分析,指出其优缺点;最后,结合边缘计算、区块链、5G等新兴技术对联邦学习的应用前景及研究热点进行展望。
中图分类号:
周传鑫, 孙奕, 汪德刚, 葛桦玮. 联邦学习研究综述[J]. 网络与信息安全学报, 2021, 7(5): 77-92.
Chuanxin ZHOU, Yi SUN, Degang WANG, Huawei GE. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021, 7(5): 77-92.
表1
通信效率算法的性能比较Table 1 Performance comparison of communication efficiency algorithms"
文献 | 压缩 | 本地优化 | 算法性能 | 算法特点 |
[30] | 弱 | 分布式计算基准算法 | ||
[2] | √ | √ | 弱 | FedAvg算法与其优化 |
[34-35] | √ | 强 | 针对FedAvg算法的优化 | |
[12] | √ | 强 | 优化筛选机制、过滤无关更新 | |
[11] | √ | 弱 | 传统压缩方法 | |
[13] | √ | √ | 强 | 传统压缩方法+算法优化 |
[38] | √ | 强 | 适应性修改压缩阈值 | |
[31] | √ | 强 | 对non-IID数据表现较好 | |
注:以FedAvg为基准,算法性能大于3倍为强压缩,小于3倍为弱压缩,“√”表示通信效率算法的类别。 |
表2
改进联邦学习隐私安全性的算法对比Table 2 Comparison of algorithms for improving the privacy and security of federated learning"
文献 | 技术结合 | 隐私保护类型 | 特点 |
[14, 21-22] | 差分隐私 | 全局隐私 | 隐藏客户端的贡献 |
[51] | 差分隐私 | 全局隐私 | 减少超参数数目 |
[23-24] | 安全多方计算 | 本地隐私 | 模型精度无损、开销大 |
[29, 54-56] | 差分隐私 | 本地隐私 | 精度略有损失 |
[45, 57] | 同态加密 | 本地隐私 | 模型精度无损 |
[58] | 差分隐私 | 本地隐私 | 应用于MEC与IoT |
[7] | 恶意检测 | 模型更新检测 | 深度学习检测恶意模型 |
[63] | 恶意检测 | 模型更新检测 | 算力资源消耗较小 |
[1] | 微众银行AI项目组. 联邦学习白皮书 V1.0[R]. 2018. |
WeBank AI Project Team. Federated learning white paper V1.0[R]. 2018. | |
[2] | MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[J]. Artificial Intelligence and Statistics, 2017: 1273-1282. |
[3] | 杨强 . AI与数据隐私保护:联邦学习的破解之道[J]. 信息安全研究, 2019,5(11): 961-965. |
YANG Q . AI and data privacy protection:the cracking method of federated learning[J]. Information Security Research. 2019,5(11): 961-965. | |
[4] | 潘碧莹, 丘海华, 张家伦 . 不同数据分布的联邦机器学习技术研究[M]. 5G 网络创新研讨会 (2019) 论文集, 2019. |
PAN B Y , QING H H , ZHANG J L . Research on federal machine learning technology with different data distribution[M]. 5G Network Innovation Seminar (2019) Proceedings, 2019. | |
[5] | YANG Q , LIU Y , CHEN T ,et al. Federated machine learning:Concept and applications[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2019,10(2): 1-19. |
[6] | HARD A , RAO K , MATHEWS R ,et al. Federated learning for mobile keyboard prediction[J]. arXiv preprint arXiv:1811.03604, 2018. |
[7] | LI S , CHENG Y , LIU Y ,et al. Abnormal client behavior detection in federated learning[J]. arXiv preprint arXiv:1910.09933, 2019. |
[8] | BONAWITZ K , IVANOV V , KREUTER B ,et al. Practical secure aggregation for privacy-preserving machine learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 1175-1191. |
[9] | MANDAL K , GONG G , LIU C . NIKE-based fast privacy-preserving high-dimensional data aggregation for mobile devices[R]. CACR Technical Report, 2018. |
[10] | CHENG K , FAN T , JIN Y ,et al. Secureboost:a lossless federated learning framework[J]. arXiv preprint arXiv:1901.08755, 2019. |
[11] | KONE?NY J , MCMAHAN H B , YU F X ,et al. Federated learning:strategies for improving communication efficiency[J]. arXiv preprint arXiv:1610.05492, 2016. |
[12] | WANG L P , WANG W , LI B . CMFL:mitigating communication overhead for federated learning[C]// 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2019: 954-964. |
[13] | CALDAS S , KONE?NY J , MC-MAHAN H B ,et al. Expanding the reach of federated learning by reducing client resource requirements[J]. arXiv preprint arXiv:1812.07210, 2018. |
[14] | BHOWMICK A , DUCHI J , FREUDIGER J ,et al. Protection against reconstruction and its applications in private federated learning[J]. arXiv preprint arXiv:1812.00984, 2018. |
[15] | CARLINI N , LIU C , KOS J ,et al. The secret sharer:Measuring unintended neural network memorization & extracting secrets[J]. arXiv preprint arXiv:1802.08232, 2018. |
[16] | FREDRIKSON M , LANTZ E , JHA S ,et al. Privacy in pharmacogenetics:an end-to-end case study of personalized warfarin dosing[C]// 23rd {USENIX} Security Symposium ({USENIX} Security 14). 2014: 17-32. |
[17] | MELIS L , SONG C , DE-CRISTOFARO E ,et al. Exploiting unintended feature leakage in collaborative learning[C]// 2019 IEEE Symposium on Security and Privacy (SP). 2019: 691-706. |
[18] | HITAJ B , ATENIESE G , PEREZ-CRUZ F , . Deep models under the GAN:information leakage from collaborative deep learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 603-618. |
[19] | GEI P J , BAUERMEISTER H , DRGE H ,et al. Inverting gradients how easy is it to break privacy in federated learning[R]. 2020. |
[20] | SONG M , WANG Z , ZHANG Z ,et al. Analyzing user-level privacy attack against federated learning[J]. IEEE Journal on Selected Areas in Communications, 2020. |
[21] | GEYER R C , KLEIN T , NABI M . Differentially private federated learning:a client level perspective[J]. arXiv preprint arXiv:1712.07557, 2017. |
[22] | MC MAHAN H B , RAMAGE D , TALWAR K ,et al. Learning differentially private recurrent language models[J]. arXiv preprint arXiv:1710.06963, 2017. |
[23] | BONAWITZ K , IVANOV V , KREUTER B ,et al. Practical secure aggregation for privacy-preserving machine learning[C]// Proceedings of the 2017 ACM Sigsac Conference on Computer and Communications Security. 2017: 1175-1191. |
[24] | MANDAL K , GONG G , LIU C . NIKE-based fast privacy-preserving high-dimensional data aggregation for mobile devices[R]. CACR Technical Report, 2018. |
[25] | CHENG K , FAN T , JIN Y ,et al. Secureboost:a lossless federated learning framework[J]. arXiv preprint arXiv:1901.08755, 2019. |
[26] | 袁勇, 王飞跃 . 区块链技术发展现状与展望[J]. 电子学报, 2016,42(4): 481-494. |
YUAN Y , WANG F Y . Blockchain:the state of the art and future trends[J]. Acta Automatica Sinica, 2016,42(4): 481-494. | |
[27] | YANG R , YU F R , SI P ,et al. Integrated blockchain and edge computing systems:a survey,some research issues and challenges[J]. IEEE Communications Surveys & Tutorials, 2019,21(2): 1508-1532. |
[28] | DONG Y , CHEN X , SHEN L ,et al. EaSTFLy:efficient and secure ternary federated learning[J]. Computers & Security, 2020,94: 1-15. |
[29] | CAO H , LIU S , ZHAO R ,et al. IFed:a novel federated learning framework for local differential privacy in power internet of things[J]. International Journal of Distributed Sensor Networks, 2020,16(5): 1-3. |
[30] | CHEN J , PAN X , MONGA R ,et al. Revisiting distributed synchronous SGD[J]. arXiv preprint arXiv:1604.00981, 2016. |
[31] | SATTLER F , WIEDEMANN S , MüLLER K R , ,et al. Robust and communication-efficient federated learning from Non-IID data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019. |
[32] | XIAO P , CHENG S , STANKOVIC V ,et al. Averaging is probably not the optimum way of aggregating parameters in federated learning[J]. Entropy, 2020,22(3): 314. |
[33] | LI T , SAHU A K , ZAHEER M ,et al. Federated optimization in heterogeneous networks[J]. arXiv preprint arXiv:1812.06127, 2018. |
[34] | LIU W , CHEN L , CHEN Y ,et al. Accelerating federated learning via momentum gradient descent[J]. IEEE Transactions on Parallel and Distributed Systems, 2020,31(8): 1754-1766. |
[35] | HUANG L , YIN Y , FU Z ,et al. LoAdaBoost:loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data[J]. PLoS ONE, 2020 15(4): 1-6. |
[36] | JIANG J , HU L , HU C ,et al. BACombo—bandwidth-aware decentralized federated learning[J]. Electronics, 2020,9(3): 440. |
[37] | REISIZADEH A , MOKHTARI A , HASSANI H ,et al. Fedpaq:a communication-efficient federated learning method with periodic averaging and quantization[C]// International Conference on Artificial Intelligence and Statistics. 2020: 2021-2031. |
[38] | LU X , LIAO Y , LIO P ,et al. Privacy-preserving asynchronous federated learning mechanism for edge network computing[J]. IEEE Access, 2020,8: 48970-48981. |
[39] | HE L , BIAN A , JAGGI M . Cola:decentralized linear learning[C]// Advances in Neural Information Processing Systems. 2018: 4536-4546. |
[40] | LALITHA A , WANG X , KILINC O ,et al. Decentralized Bayesian learning over graphs[J]. arXiv preprint arXiv:1905.10466, 2019. |
[41] | LIN T , STICH S U , PATEL K K ,et al. Don't use large mini-batches,use local SGD[J]. arXiv preprint arXiv:1808.07217, 2018. |
[42] | LIU L , ZHANG J , SONG S H ,et al. Edge-assisted hierarchical federated learning with non-iid data[J]. arXiv preprint arXiv:1905.06641, 2019. |
[43] | SHARMA P K , PARK J H , CHO K . Blockchain and federated learning-based distributed computing defence framework for sustainable society[J]. Sustainable Cities and Society, 2020:102220. |
[44] | ZHANG J , ZHAO Y , WANG J ,et al. FedMEC:improving efficiency of differentially private federated learning via mobile edge computing[J]. Mobile Networks and Applications, 2020: 1-13. |
[45] | 刘俊旭, 孟小峰 . 机器学习的隐私保护研究综述[J]. 计算机研究与发展, 2020,57(2): 346. |
LIU J X , MENG X F . A survey of research on privacy protection in machine learning[J]. Computer Research and Development, 2020,57(2): 346. | |
[46] | DWORK C , MC-SHERRY F , NISSIM K ,et al. Calibrating noise to sensitivity in private data analysis[C]// Theory of Cryptography Conference. 2006: 265-284. |
[47] | 叶青青, 孟小峰, 朱敏杰 ,等. 本地化差分隐私研究综述[J]. 软件学报, 2018,29(7): 159-183. |
YE Q Q , MENG X F , ZHU M J ,et al. Survey of localized differential privacy research[J]. Journal of Software, 2018,29(7): 159-183. | |
[48] | 苏冠通, 徐茂桐 . 安全多方计算技术与应用综述[J]. 信息通信技术与政策, 2019(5): 19-22. |
SU G T , XU M T . Survey of secure multiparty computing technology and application[J]. Information and Communication Technol ogy and Policy, 2019(5): 19-22. | |
[49] | DOLEV D , YAO A . On the security of public key protocols[J]. IEEE Transactions on Information Theory, 1983,29(2): 198-208. |
[50] | RIVEST R L , SHAMIR A , ADLEMAN L . A method for obtaining digital signatures and public-key cryptosystems[J]. Communications of the ACM, 1978,21(2): 120-126. |
[51] | THAKKAR O , ANDREW G , MC-MAHAN H B , . Differentially private learning with adaptive clipping[J]. arXiv preprint arXiv:1905.03871, 2019. |
[52] | HAO M , LI H , LUO X ,et al. Efficient and privacy-enhanced federated learning for industrial artificial intelligence[J]. IEEE Transactions on Industrial Informatics, 2019,16(10): 6532-6542. |
[53] | AONO Y , HAYASHI T , WANG L ,et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2017,13(5): 1333-1345. |
[54] | LIU X , LI H , XU G ,et al. Adaptive privacy-preserving federated learning[J]. Peer to Peer Networking and Applications, 2020,13: 2356-2366. |
[55] | HUANG X , DING Y , JIANG Z L ,et al. DP-FL:a novel differentially private federated learning framework for the unbalanced data[J]. World Wide Web, 2020: 1-17. |
[56] | WEI K , LI J , DING M ,et al. Federated learning with differential privacy:algorithms and performance analysis[J]. IEEE Transactions on Information Forensics and Security, 2020,15: 3454-3469. |
[57] | XU G , LI H , LIU S ,et al. VerifyNet:secure and verifiable federated learning[J]. IEEE Transactions on Information Forensics and Security, 2019,15: 911-926. |
[58] | LU Y , HUANG X , DAI Y ,et al. Differentially private asynchronous federated learning for mobile edge computing in urban informatics[J]. IEEE Transactions on Industrial Informatics, 2019. |
[59] | LU Y , HUANG X , DAI Y ,et al. Federated learning for data privacy preservation in vehicular cyber-physical systems[J]. IEEE Network, 2020,34(3): 50-56. |
[60] | HU R , GUO Y , LI H ,et al. Personalized federated learning with differential privacy[J]. IEEE Internet of Things Journal, 2020,7(10): 9530-9539. |
[61] | FANG M , CAO X , JIA J ,et al. Local model poisoning attacks to Byzantine-robust federated learning[J]. arXiv preprint arXiv:1911.11815, 2019. |
[62] | ZHAO Y , CHEN J , ZHANG J ,et al. Detecting and mitigating poisoning attacks in federated learning using generative adversarial networks[J]. Concurrency and Computation:Practice and Experience, 2020: 1-2. |
[63] | KANG J , XIONG Z , NIYATO D ,et al. Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory[J]. IEEE Internet of Things Journal, 2019,6(6): 10700-10714. |
[64] | FUNG C , YOON C J M , BESCHASTNIKH I . Mitigating sybils in federated learning poisoning[J]. arXiv preprint arXiv:1808.04866, 2018. |
[65] | CHEN Y , LUO F , LI T ,et al. A training-integrity privacy-preserving federated learning scheme with trusted execution environment[J]. Information Sciences, 2020,522: 69-79. |
[66] | LYU L , YU J , NANDAKUMAR K ,et al. Towards fair and privacy-preserving federated deep models[J]. IEEE Transactions on Parallel and Distributed Systems, 2020,31(11): 2524-2541. |
[67] | MAJEED U , HONG C S . FLchain:federated learning via MEC-enabled blockchain network[C]// 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS). 2019: 1-4. |
[68] | ARACHCHIGE P C M , BERTOK P , KHALIL I ,et al. A trustworthy privacy preserving framework for machine learning in industrial iot systems[J]. IEEE Transactions on Industrial Informatics, 2020,16(9): 6092-6102. |
[69] | LU Y , HUANG X , ZHANG K ,et al. Blockchain empowered asynchronous federated learning for secure data sharing in internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2020,69(4): 4298-4311. |
[70] | POKHREL S R . Towards efficient and reliable federated learning using blockchain for autonomous vehicles[J]. Computer Networks, 2020:107431. |
[71] | KIM H , PARK J , BENNIS M ,et al. On-device federated learning via blockchain and its latency analysis[J]. arXiv preprint arXiv:1808.03949, 2018. |
[72] | KANG J , XIONG Z , NIYATO D ,et al. Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory[J]. IEEE Internet of Things Journal, 2019,6(6): 10700-10714. |
[73] | WENG J , WENG J , ZHANG J ,et al. Deepchain:auditable and privacy-preserving deep learning with blockchain-based incentive[J]. IEEE Transactions on Dependable and Secure Computing, 2019. |
[74] | KIM Y J , HONG C S . Blockchain-based node-aware dynamic weighting methods for improving federated learning performance[C]// 20th Asia-Pacific Network Operations and Management Symposium (APNOMS). 2019: 1-4. |
[75] | ZHAN Y , LI P , QU Z ,et al. A learning-based incentive mechanism for federated learning[J]. IEEE Internet of Things Journal, 2020,7(7): 6360-6368. |
[76] | PREUVENEERS D , RIMMER V , TSINGENOPOULOS I ,et al. Chained anomaly detection models for federated learning:an intrusion detection case study[J]. Applied Sciences, 2018,8(12): 2663. |
[77] | ZHU X , LI H , YU Y . Blockchain-based privacy preserving deep learning[C]// International Conference on Information Security and Cryptology. 2018: 370-383. |
[78] | QU Y , GAO L , LUAN T H ,et al. Decentralized privacy using blockchain-enabled federated learning in fog computing[J]. IEEE Internet of Things Journal, 2020,7(6): 5171-5183. |
[79] | LIU Y , PENG J , KANG J ,et al. A secure federated learning framework for 5G networks[J]. arXiv preprint arXiv:2005.05752, 2020. |
[80] | KAIROUZ P , MC-MAHAN H B , AVENT B ,et al. Advances and open problems in federated learning[J]. arXiv preprint arXiv:1912.04977, 2019. |
[81] | LI T , SAHU A K , TALWALKAR A ,et al. Federated learning:challenges,methods,and future directions[J]. arXiv preprint arXiv:1908.07873, 2019. |
[82] | SMITH V , CHIANG C K , SANJABI M ,et al. Federated multi-task learning[C]// Advances in Neural Information Processing Systems. 2017: 4424-4434. |
[83] | ANG F , CHEN L , ZHAO N ,et al. Robust federated learning with noisy communication[J]. IEEE Transactions on Communications, 2020. |
[84] | NIKNAM S , DHILLON H S , REED J H . Federated learning for wireless communications:motivation,opportunities,and challenges[J]. IEEE Communications Magazine, 2020,58(6): 46-51. |
[85] | REN J , WANG H , HOU T ,et al. Federated learning-based computation offloading optimization in edge computing-supported internet of things[J]. IEEE Access, 2019,7: 69194-69201. |
[86] | WANG X , HAN Y , WANG C ,et al. In-edge AI:intelligentizing mobile edge computing,caching and communication by federated learning[J]. IEEE Network, 2019,33(5): 156-165. |
[87] | WANG S , TUOR T , SALONIDIS T ,et al. Adaptive federated learning in resource constrained edge computing systems[J]. IEEE Journal on Selected Areas in Communications, 2019,37(6): 1205-1221. |
[1] | 陈赛特, 李卫海, 姚远志, 俞能海. 轻量级K匿名增量近邻查询位置隐私保护算法[J]. 网络与信息安全学报, 2023, 9(3): 60-72. |
[2] | 余锋, 林庆新, 林晖, 汪晓丁. 基于生成对抗网络的隐私增强联邦学习方案[J]. 网络与信息安全学报, 2023, 9(3): 113-122. |
[3] | 陈晋音, 李荣昌, 黄国瀚, 刘涛, 郑海斌, 程瑶. 纵向联邦学习方法及其隐私和安全综述[J]. 网络与信息安全学报, 2023, 9(2): 1-20. |
[4] | 蔡召, 荆涛, 任爽. 以太坊钓鱼诈骗检测技术综述[J]. 网络与信息安全学报, 2023, 9(2): 21-32. |
[5] | 肖敏, 毛发英, 黄永洪, 曹云飞. 基于属性签名的车载网匿名信任管理方案[J]. 网络与信息安全学报, 2023, 9(2): 33-45. |
[6] | 许建龙, 林健, 黎宇森, 熊智. 分布式用户隐私保护可调节的云服务个性化QoS预测模型[J]. 网络与信息安全学报, 2023, 9(2): 70-80. |
[7] | 王贺立, 闫巧. 基于交易记录特征的自私挖矿检测方案[J]. 网络与信息安全学报, 2023, 9(2): 104-114. |
[8] | 余北缘, 任珊瑶, 刘建伟. 区块链资产窃取攻击与防御技术综述[J]. 网络与信息安全学报, 2023, 9(1): 1-17. |
[9] | 孙哲, 宁洪, 殷丽华, 方滨兴. 基于教学实训靶场的“数据隐私保护”课程建设初探[J]. 网络与信息安全学报, 2023, 9(1): 178-188. |
[10] | 唐飞, 甘宁, 阳祥贵, 王金洋. 基于区块链与国密SM9的抗恶意KGC无证书签名方案[J]. 网络与信息安全学报, 2022, 8(6): 9-19. |
[11] | 白雪, 秦宝东, 郭瑞, 郑东. 基于SM2的两方协作盲签名协议[J]. 网络与信息安全学报, 2022, 8(6): 39-51. |
[12] | 肖敏, 姚涛, 刘媛妮, 黄永洪. 具有隐私保护的动态高效车载云管理方案[J]. 网络与信息安全学报, 2022, 8(6): 70-83. |
[13] | 卢晨昕, 陈兵, 丁宁, 陈立全, 吴戈. 具有紧凑标签的基于身份匿名云审计方案[J]. 网络与信息安全学报, 2022, 8(6): 156-168. |
[14] | 明盛智, 朱建明, 隋智源, 张娴. 信息增值机制下在线医疗隐私保护策略[J]. 网络与信息安全学报, 2022, 8(6): 169-177. |
[15] | 应作斌, 方一晨, 张怡文. 动态聚合权重的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022, 8(5): 56-65. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||
|