支持多数不规则用户的隐私保护联邦学习框架

doi:10.11959/j.issn.2096-109x.2022011

网络与信息安全学报 ›› 2022, Vol. 8 ›› Issue (1): 139-150.doi: 10.11959/j.issn.2096-109x.2022011

支持多数不规则用户的隐私保护联邦学习框架

陈前昕¹^,², 毕仁万¹^,², 林劼¹, 金彪¹, 熊金波¹^,²

¹ 福建师范大学计算机与网络空间安全学院，福建福州350117
² 福建师范大学福建省网络安全与密码技术重点实验室，福建福州 350007

修回日期:2022-01-05 出版日期:2022-02-15 发布日期:2022-02-01
作者简介:陈前昕（1996− ），男，福建泉州人，福建师范大学硕士生，主要研究方向为安全深度学习、隐私保护技术
毕仁万（1996− ），男，湖南常德人，福建师范大学博士生，主要研究方向为安全深度学习、安全多方计算
林劼（1972− ），男，福建三明人，博士，福建师范大学教授，主要研究方向为机器学习、生物信息学
金彪（1985− ），男，安徽六安人，博士，福建师范大学高级实验师，主要研究方向为信息安全、数据挖掘
熊金波（1981−），男，湖南益阳人，博士，福建师范大学教授，主要研究方向为安全深度学习、移动群智感知、隐私保护技术
基金资助:
国家自然科学基金(61872088);国家自然科学基金(61872090);国家自然科学基金(U1905211);福建省自然科学基金(2019J01276)

Privacy-preserving federated learning framework with irregular-majority users

Qianxin CHEN¹^,², Renwan BI¹^,², Jie LIN¹, Biao JIN¹, Jinbo XIONG¹^,²

¹ College of Computer and Cyber Security, Fujian Normal University, Fuzhou 350117, China
² Fujian Provincial Key Laboratory of Network Security and Cryptology, Fujian Normal University, Fuzhou 350007, China

Revised:2022-01-05 Online:2022-02-15 Published:2022-02-01
Supported by:
The National Natural Science Foundation of China(61872088);The National Natural Science Foundation of China(61872090);The National Natural Science Foundation of China(U1905211);The Natural Science Foundation of Fujian Province(2019J01276)

摘要/Abstract

摘要：

针对联邦学习存在处理大多数不规则用户易引起聚合效率降低，以及采用明文通信导致参数隐私泄露的问题，基于设计的安全除法协议构建针对不规则用户鲁棒的隐私保护联邦学习框架。该框架通过将模型相关计算外包给两台边缘服务器以减小采用同态加密产生的高额计算开销，不仅允许模型及其相关信息以密文形式在边缘服务器上进行密文聚合，还支持用户在本地进行模型可靠性计算以减小传统方法采用安全乘法协议造成的额外通信开销。在该框架的基础上，为更精准评估模型的泛化性能，用户完成本地模型参数更新后，利用边缘服务器下发的验证集与本地持有的验证集联合计算模型损失值，并结合损失值历史信息动态更新模型可靠性以作为模型权重。进一步，在模型可靠性先验知识指导下进行模型权重缩放，将密文模型与密文权重信息交由边缘服务器对全局模型参数进行聚合更新，保证全局模型变化主要由高质量数据用户贡献，提高收敛速度。通过HybridArgument模型进行安全性分析，论证表明PPRFL（privacy-preserving robust federated learning）可以有效保护模型参数以及包括用户可靠性等中间交互参数的隐私。实验结果表明，当联邦聚合任务中的所有参与方均为不规则用户时，PPRFL 方案准确率仍然能达到 92%，收敛效率较 PPFDL （privacy-preserving federated deep learning with irregular users）提高1.4倍；当联邦聚合任务中80%用户持有的训练数据都为噪声数据时，PPRFL方案准确率仍然能达到89%，收敛效率较PPFDL提高2.3倍。

关键词: 联邦学习, 隐私保护, 安全聚合, 大多数不规则用户, 安全除法协议

Abstract:

In response to the existing problems that the federated learning might lead to the reduction of aggregation efficiency by handing the majority of irregular users and the leak of parameter privacy by adopting plaintext communication, a framework of privacy-preserving robust federated learning was proposed for ensuring the robustness of the irregular user based on the designed security division protocol.PPRFL could enable the model and its related information to aggregate in ciphertext on the edge server facilitate users to calculate the model reliability locally for reducing the additional communication overhead caused by the adoption of the security multiplication protocol in conventional methods, apart from lowering the high computational overhead resulted from homomorphic encryption with outsourcing computing to two edge servers.Based on this, user could calculate the loss value of the model through jointly using the verification sets issued by the edge server and that held locally after parameter updating of the local model.Then the model reliability could be dynamically updated as the model weight together with the historic information of the loss value.Further, the model weight was scaled under the guidance of prior knowledge, and the ciphertext model and ciphertext weight information are sent to the edge server to aggregate and update the global model parameters, ensuring that global model changes are contributed by high-quality data users, and improving the convergence speed.Through the security analysis of the Hybrid Argument model, the demonstration shows that PPRFL can effectively protect the privacy of model parameters and intermediate interaction parameters including user reliability.The experimental results show that the PPRFL scheme could still achieve the accuracy of 92% when all the participants in the federated aggregation task are irregular users, with the convergence efficiency 1.4 times higher than that of the PPFDL.Besides, the PPRFL scheme could still reach the accuracy of 89% when training data possessed by 80% of the users in the federated aggregation task were noise data, with the convergence efficiency 2.3 times higher than that of the PPFDL.

Key words: federated learning, privacy-preserving, secure aggregation, irregular-majority users, security division protocol

中图分类号:

TP309

陈前昕, 毕仁万, 林劼, 金彪, 熊金波. 支持多数不规则用户的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022, 8(1): 139-150.

Qianxin CHEN, Renwan BI, Jie LIN, Biao JIN, Jinbo XIONG. Privacy-preserving federated learning framework with irregular-majority users[J]. Chinese Journal of Network and Information Security, 2022, 8(1): 139-150.

图/表 10

图1

表1

表2

图2

图3

表3

图4

图5

图6

表4

参考文献 37

[1]	LI T , SAHU A K , TALWALKAR A ,et al. Federated learning:challenges,methods,and future directions[J]. IEEE Signal Processing Magazine, 2020,37(3): 50-60.
[2]	YANG Q , LIU Y , CHEN T J ,et al. Federated machine learning[J]. ACM Transactions on Intelligent Systems and Technology, 2019,10(2): 1-19.
[3]	梁应敞, 谭俊杰, 智能无线通信技术研究概况[J]. 通信学报, 2020,41(7): 1-17.
	LIANG Y C , TAN J J , NIYATO D . Overview on intelligent wireless communication technology[J]. Journal on Communications, 2020,41(7): 1-17.
[4]	杨强 . AI与数据隐私保护：联邦学习的破解之道[J]. 信息安全研究, 2019,5(11): 961-965.
	YANG Q . AI and data privacy protection:the way to federated learning[J]. Journal of Information Security Research, 2019,5(11): 961-965.
[5]	谭清尹, 曾颖明, 韩叶 ,等. 神经网络后门攻击研究[J]. 网络与信息安全学报, 2021,7(3): 46-58.
	TAN Q Y , ZENG Y M , HAN Y ,et al. Survey on backdoor attacks targeted on neural network[J]. Chinese Journal of Network and Information Security, 2021,7(3): 46-58.
[6]	MELIS L , SONG C Z , DE CRISTOFARO E ,et al. Exploiting unintended feature leakage in collaborative learning[C]// Proceedings of 2019 IEEE Symposium on Security and Privacy. Piscataway:IEEE Press, 2019: 691-706.
[7]	周传鑫, 孙奕, 汪德刚 ,等. 联邦学习研究综述[J]. 网络与信息安全学报, 2021,7(5): 77-92.
	ZHOU C X , SUN Y , WANG D G ,et al. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021,7(5): 77-92.
[8]	HITAJ B , ATENIESE G , PEREZ-CRUZ F , . Deep models under the GAN:information leakage from collaborative deep learning[C]// Proceedings of CCS '17:Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 603-618.
[9]	MOTHUKURI V , PARIZI R M , POURIYEH S ,et al. A survey on security and privacy of federated learning[J]. Future Generation Computer Systems, 2021,115: 619-640.
[10]	WAGH S , GUPTA D , CHANDRAN N . SecureNN:3-party secure computation for neural network training[J]. Proceedings on Privacy Enhancing Technologies, 2019,2019(3): 26-49.
[11]	XU R H , BARACALDO N , ZHOU Y ,et al. HybridAlpha:an efficient approach for privacy-preserving federated learning[C]// Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. 2019: 13-23.
[12]	方晨, 郭渊博, 王一丰 ,等. 基于区块链和联邦学习的边缘计算隐私保护方法[J]. 通信学报, 2021,42(11): 28-40.
	FANG C , GUO Y B , WANG Y F ,et al. Edge computing privacy protection method based on blockchain and federated learning[J]. Journal on Communications, 2021,42(11): 28-40.
[13]	KANAGAVELU R , LI Z X , SAMSUDIN J ,et al. Two-phase multi-party computation enabled privacy-preserving federated learning[C]// Proceedings of 2020 20th IEEE/ACM International Symposium on Cluster,Cloud and Internet Computing (CCGRID). 2020: 410-419.
[14]	董业, 侯炜, 陈小军 ,等. 基于秘密分享和梯度选择的高效安全联邦学习[J]. 计算机研究与发展, 2020,57(10): 2241-2250.
	DONG Y , HOU W , CHEN X J ,et al. Efficient and secure federated learning based on secret sharing and gradients selection[J]. Journal of Computer Research and Development, 2020,57(10): 2241-2250.
[15]	PHONG L T , AONO Y , HAYASHI T ,et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2018,13(5): 1333-1345.
[16]	张泽辉, 富瑶, 高铁杠 . 支持数据隐私保护的联邦深度神经网络模型研究[J]. 自动化学报, 2020.
	ZHANG Z H , FU Y , GAO T G . Research on federated deep neural network model for data privacy protection[J]. Acta Automatica Sinica, 2020.
[17]	ZHAO L C , WANG Q , ZOU Q ,et al. Privacy-preserving collaborative deep learning with unreliable participants[J]. IEEE Transactions on Information Forensics and Security, 2020,15: 1486-1500.
[18]	JAYARAMAN B , EVANS D . Evaluating differentially private machine learning in practice[C]// Proceedings of the28th USENIX Security Symposium. 2019: 1895-1912.
[19]	成艺 . 联合学习环境下保护隐私的数据聚合技术研究[D]. 成都:电子科技大学, 2020: 17-45.
	CHENG Y . Research on data aggregation technology based on privacy-preserving in federated learning[D]. Chengdu:University of Electronic Science and Technology of China, 2020: 17-45.
[20]	SHAMIR A . How to share a secret[J]. Communications of the ACM, 1979,22(11): 612-613.
[21]	XU G W , LI H W , ZHANG Y ,et al. Privacy-preserving federated deep learning with irregular users[J]. IEEE Transactions on Dependable and Secure Computing, 2020,(99):1.
[22]	ZHENG Y F , DUAN H Y , WANG C . Learning the truth privately and confidently:encrypted confidence-aware truth discovery in mobile crowdsensing[J]. IEEE Transactions on Information Forensics and Security, 2018,13(10): 2475-2489.
[23]	TIAN Y L , LI T , XIONG J B ,et al. A blockchain-based machine learning framework for edge services in IoT[J]. IEEE Transactions on Industrial Informatics, 2022,18(3): 1918-1929.
[24]	XIONG J B , BI R W , ZHAO M F ,et al. Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles[J]. IEEE Wireless Communications, 2020,27(3): 24-30.
[25]	BONAWITZ K , EICHNER H , GRIESKAMP W ,et al. Towards federated learning at scale:system design[J]. arXiv preprint arXiv:1902.01046, 2019.
[26]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017: 1273-1282.
[27]	PAILLIER P , . Public-key cryptosystems based on composite degree residuosity classes[C]// Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques. 1999: 223-238.
[28]	ACAR A , AKSU H , ULUAGAC A S ,et al. A survey on homomorphic encryption schemes[J]. ACM Computing Surveys, 2019,51(4): 1-35.
[29]	CANETTI R , FEIGE U , GOLDREICH O ,et al. Adaptively secure multi-party computation[C]// Proceedings of the twenty-eighth annual ACM Symposium on Theory of Computing. 1996: 639-648.
[30]	HENDERSON M , THOMSON B , WILLIAMS J D . The third dialog state tracking challenge[C]// Proceedings of 2014 IEEE Spoken Language Technology Workshop. 2014: 324-329.
[31]	BUDZIANOWSKI P , WEN T H , TSENG B H ,et al. MultiWOZ A large-scale multi-domain wizard-of-o_zdataset for task-oriented dialogue modelling[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 5016-5026.
[32]	SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning[C]// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015: 1310-1321.
[33]	KANG J W , XIONG Z H , NIYATO D ,et al. Toward secure blockchain-enabled Internet of vehicles:optimizing consensus management using reputation and contract theory[J]. IEEE Transactions on Vehicular Technology, 2019,68(3): 2906-2920.
[34]	KANG J W , XIONG Z H , NIYATO D ,et al. Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory[J]. IEEE Internet of Things Journal, 2019,6(6): 10700-10714.
[35]	KANG J W , XIONG Z H , NIYATO D ,et al. Reliable federated learning for mobile networks[J]. IEEE Wireless Communications, 2020,27(2): 72-80.
[36]	CATALANO D , FIORE D . Using linearly-homomorphic encryption to evaluate degree-2 functions on encrypted data[C]// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015: 1518-1529.
[37]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.

方案	保护模型参数	保护用户可靠性	保护聚合参数	用户退出鲁棒性	模型收敛效率
ecProbe^[17]	√	×	×	×	×
SAHPP^[19]	√	√	√	√	×
PPFDL^[21]	√	√	√	√	×
PPRFL	√	√	√	√	√
注：“√”表示满足对应功能，“×”表示不满足。

方案	用户与第三方	用户与服务器S₀	服务器S₀与S₁
SAHPP^[19]	O(n)	O(n²+nm)	—
PPFDL^[21]	—	O (nm)	O (κ(n+m))
PPRFL	—	O (nm)	O (m)

方案	采用STT前	采用STT后
SAHPP^[19]	O (nm)
PPFDL^[21]	O (2κnm)	O (2κ(n+m))
PPRFL	O (n+nm)	O (n+m)

协议	T_OT	T_GC	CMult	Mult	Enc	Dec
SecDiv^[21]	1	1	3	—	4	2
本文协议	—	—	3	1	3	2

支持多数不规则用户的隐私保护联邦学习框架

Privacy-preserving federated learning framework with irregular-majority users

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 37

相关文章 15

Metrics

推荐阅读 0

[1]	陈赛特, 李卫海, 姚远志, 俞能海. 轻量级K匿名增量近邻查询位置隐私保护算法[J]. 网络与信息安全学报, 2023, 9(3): 60-72.
[2]	余锋, 林庆新, 林晖, 汪晓丁. 基于生成对抗网络的隐私增强联邦学习方案[J]. 网络与信息安全学报, 2023, 9(3): 113-122.
[3]	陈晋音, 李荣昌, 黄国瀚, 刘涛, 郑海斌, 程瑶. 纵向联邦学习方法及其隐私和安全综述[J]. 网络与信息安全学报, 2023, 9(2): 1-20.
[4]	肖敏, 毛发英, 黄永洪, 曹云飞. 基于属性签名的车载网匿名信任管理方案[J]. 网络与信息安全学报, 2023, 9(2): 33-45.
[5]	许建龙, 林健, 黎宇森, 熊智. 分布式用户隐私保护可调节的云服务个性化QoS预测模型[J]. 网络与信息安全学报, 2023, 9(2): 70-80.
[6]	孙哲, 宁洪, 殷丽华, 方滨兴. 基于教学实训靶场的“数据隐私保护”课程建设初探[J]. 网络与信息安全学报, 2023, 9(1): 178-188.
[7]	白雪, 秦宝东, 郭瑞, 郑东. 基于SM2的两方协作盲签名协议[J]. 网络与信息安全学报, 2022, 8(6): 39-51.
[8]	肖敏, 姚涛, 刘媛妮, 黄永洪. 具有隐私保护的动态高效车载云管理方案[J]. 网络与信息安全学报, 2022, 8(6): 70-83.
[9]	卢晨昕, 陈兵, 丁宁, 陈立全, 吴戈. 具有紧凑标签的基于身份匿名云审计方案[J]. 网络与信息安全学报, 2022, 8(6): 156-168.
[10]	明盛智, 朱建明, 隋智源, 张娴. 信息增值机制下在线医疗隐私保护策略[J]. 网络与信息安全学报, 2022, 8(6): 169-177.
[11]	应作斌, 方一晨, 张怡文. 动态聚合权重的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022, 8(5): 56-65.
[12]	张娴, 朱建明, 隋智源, 明盛智. 数字货币交易匿名性与监管的博弈分析[J]. 网络与信息安全学报, 2022, 8(5): 150-157.
[13]	刘峰, 杨杰, 齐佳音. 区块链密码学隐私保护技术综述[J]. 网络与信息安全学报, 2022, 8(4): 29-44.
[14]	金琳, 田有亮. 基于区块链的多权限属性隐藏电子病历共享方案[J]. 网络与信息安全学报, 2022, 8(4): 66-76.
[15]	张伟成, 卫红权, 刘树新, 普黎明. 5G移动边缘计算场景下的快速切换认证方案[J]. 网络与信息安全学报, 2022, 8(3): 154-168.