支持多数不规则用户的隐私保护联邦学习框架

doi:10.11959/j.issn.2096-109x.2022011

Abstract

Abstract:

In response to the existing problems that the federated learning might lead to the reduction of aggregation efficiency by handing the majority of irregular users and the leak of parameter privacy by adopting plaintext communication, a framework of privacy-preserving robust federated learning was proposed for ensuring the robustness of the irregular user based on the designed security division protocol.PPRFL could enable the model and its related information to aggregate in ciphertext on the edge server facilitate users to calculate the model reliability locally for reducing the additional communication overhead caused by the adoption of the security multiplication protocol in conventional methods, apart from lowering the high computational overhead resulted from homomorphic encryption with outsourcing computing to two edge servers.Based on this, user could calculate the loss value of the model through jointly using the verification sets issued by the edge server and that held locally after parameter updating of the local model.Then the model reliability could be dynamically updated as the model weight together with the historic information of the loss value.Further, the model weight was scaled under the guidance of prior knowledge, and the ciphertext model and ciphertext weight information are sent to the edge server to aggregate and update the global model parameters, ensuring that global model changes are contributed by high-quality data users, and improving the convergence speed.Through the security analysis of the Hybrid Argument model, the demonstration shows that PPRFL can effectively protect the privacy of model parameters and intermediate interaction parameters including user reliability.The experimental results show that the PPRFL scheme could still achieve the accuracy of 92% when all the participants in the federated aggregation task are irregular users, with the convergence efficiency 1.4 times higher than that of the PPFDL.Besides, the PPRFL scheme could still reach the accuracy of 89% when training data possessed by 80% of the users in the federated aggregation task were noise data, with the convergence efficiency 2.3 times higher than that of the PPFDL.

Key words: federated learning, privacy-preserving, secure aggregation, irregular-majority users, security division protocol

CLC Number:

TP309

Qianxin CHEN, Renwan BI, Jie LIN, Biao JIN, Jinbo XIONG. Privacy-preserving federated learning framework with irregular-majority users[J]. Chinese Journal of Network and Information Security, 2022, 8(1): 139-150.

Figures/Tables 10

References 37

[1]	LI T , SAHU A K , TALWALKAR A ,et al. Federated learning:challenges,methods,and future directions[J]. IEEE Signal Processing Magazine, 2020,37(3): 50-60.
[2]	YANG Q , LIU Y , CHEN T J ,et al. Federated machine learning[J]. ACM Transactions on Intelligent Systems and Technology, 2019,10(2): 1-19.
[3]	梁应敞, 谭俊杰, 智能无线通信技术研究概况[J]. 通信学报, 2020,41(7): 1-17.
	LIANG Y C , TAN J J , NIYATO D . Overview on intelligent wireless communication technology[J]. Journal on Communications, 2020,41(7): 1-17.
[4]	杨强 . AI与数据隐私保护：联邦学习的破解之道[J]. 信息安全研究, 2019,5(11): 961-965.
	YANG Q . AI and data privacy protection:the way to federated learning[J]. Journal of Information Security Research, 2019,5(11): 961-965.
[5]	谭清尹, 曾颖明, 韩叶 ,等. 神经网络后门攻击研究[J]. 网络与信息安全学报, 2021,7(3): 46-58.
	TAN Q Y , ZENG Y M , HAN Y ,et al. Survey on backdoor attacks targeted on neural network[J]. Chinese Journal of Network and Information Security, 2021,7(3): 46-58.
[6]	MELIS L , SONG C Z , DE CRISTOFARO E ,et al. Exploiting unintended feature leakage in collaborative learning[C]// Proceedings of 2019 IEEE Symposium on Security and Privacy. Piscataway:IEEE Press, 2019: 691-706.
[7]	周传鑫, 孙奕, 汪德刚 ,等. 联邦学习研究综述[J]. 网络与信息安全学报, 2021,7(5): 77-92.
	ZHOU C X , SUN Y , WANG D G ,et al. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021,7(5): 77-92.
[8]	HITAJ B , ATENIESE G , PEREZ-CRUZ F , . Deep models under the GAN:information leakage from collaborative deep learning[C]// Proceedings of CCS '17:Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 603-618.
[9]	MOTHUKURI V , PARIZI R M , POURIYEH S ,et al. A survey on security and privacy of federated learning[J]. Future Generation Computer Systems, 2021,115: 619-640.
[10]	WAGH S , GUPTA D , CHANDRAN N . SecureNN:3-party secure computation for neural network training[J]. Proceedings on Privacy Enhancing Technologies, 2019,2019(3): 26-49.
[11]	XU R H , BARACALDO N , ZHOU Y ,et al. HybridAlpha:an efficient approach for privacy-preserving federated learning[C]// Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security. 2019: 13-23.
[12]	方晨, 郭渊博, 王一丰 ,等. 基于区块链和联邦学习的边缘计算隐私保护方法[J]. 通信学报, 2021,42(11): 28-40.
	FANG C , GUO Y B , WANG Y F ,et al. Edge computing privacy protection method based on blockchain and federated learning[J]. Journal on Communications, 2021,42(11): 28-40.
[13]	KANAGAVELU R , LI Z X , SAMSUDIN J ,et al. Two-phase multi-party computation enabled privacy-preserving federated learning[C]// Proceedings of 2020 20th IEEE/ACM International Symposium on Cluster,Cloud and Internet Computing (CCGRID). 2020: 410-419.
[14]	董业, 侯炜, 陈小军 ,等. 基于秘密分享和梯度选择的高效安全联邦学习[J]. 计算机研究与发展, 2020,57(10): 2241-2250.
	DONG Y , HOU W , CHEN X J ,et al. Efficient and secure federated learning based on secret sharing and gradients selection[J]. Journal of Computer Research and Development, 2020,57(10): 2241-2250.
[15]	PHONG L T , AONO Y , HAYASHI T ,et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2018,13(5): 1333-1345.
[16]	张泽辉, 富瑶, 高铁杠 . 支持数据隐私保护的联邦深度神经网络模型研究[J]. 自动化学报, 2020.
	ZHANG Z H , FU Y , GAO T G . Research on federated deep neural network model for data privacy protection[J]. Acta Automatica Sinica, 2020.
[17]	ZHAO L C , WANG Q , ZOU Q ,et al. Privacy-preserving collaborative deep learning with unreliable participants[J]. IEEE Transactions on Information Forensics and Security, 2020,15: 1486-1500.
[18]	JAYARAMAN B , EVANS D . Evaluating differentially private machine learning in practice[C]// Proceedings of the28th USENIX Security Symposium. 2019: 1895-1912.
[19]	成艺 . 联合学习环境下保护隐私的数据聚合技术研究[D]. 成都:电子科技大学, 2020: 17-45.
	CHENG Y . Research on data aggregation technology based on privacy-preserving in federated learning[D]. Chengdu:University of Electronic Science and Technology of China, 2020: 17-45.
[20]	SHAMIR A . How to share a secret[J]. Communications of the ACM, 1979,22(11): 612-613.
[21]	XU G W , LI H W , ZHANG Y ,et al. Privacy-preserving federated deep learning with irregular users[J]. IEEE Transactions on Dependable and Secure Computing, 2020,(99):1.
[22]	ZHENG Y F , DUAN H Y , WANG C . Learning the truth privately and confidently:encrypted confidence-aware truth discovery in mobile crowdsensing[J]. IEEE Transactions on Information Forensics and Security, 2018,13(10): 2475-2489.
[23]	TIAN Y L , LI T , XIONG J B ,et al. A blockchain-based machine learning framework for edge services in IoT[J]. IEEE Transactions on Industrial Informatics, 2022,18(3): 1918-1929.
[24]	XIONG J B , BI R W , ZHAO M F ,et al. Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles[J]. IEEE Wireless Communications, 2020,27(3): 24-30.
[25]	BONAWITZ K , EICHNER H , GRIESKAMP W ,et al. Towards federated learning at scale:system design[J]. arXiv preprint arXiv:1902.01046, 2019.
[26]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017: 1273-1282.
[27]	PAILLIER P , . Public-key cryptosystems based on composite degree residuosity classes[C]// Proceedings of the International Conference on the Theory and Applications of Cryptographic Techniques. 1999: 223-238.
[28]	ACAR A , AKSU H , ULUAGAC A S ,et al. A survey on homomorphic encryption schemes[J]. ACM Computing Surveys, 2019,51(4): 1-35.
[29]	CANETTI R , FEIGE U , GOLDREICH O ,et al. Adaptively secure multi-party computation[C]// Proceedings of the twenty-eighth annual ACM Symposium on Theory of Computing. 1996: 639-648.
[30]	HENDERSON M , THOMSON B , WILLIAMS J D . The third dialog state tracking challenge[C]// Proceedings of 2014 IEEE Spoken Language Technology Workshop. 2014: 324-329.
[31]	BUDZIANOWSKI P , WEN T H , TSENG B H ,et al. MultiWOZ A large-scale multi-domain wizard-of-o_zdataset for task-oriented dialogue modelling[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018: 5016-5026.
[32]	SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning[C]// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015: 1310-1321.
[33]	KANG J W , XIONG Z H , NIYATO D ,et al. Toward secure blockchain-enabled Internet of vehicles:optimizing consensus management using reputation and contract theory[J]. IEEE Transactions on Vehicular Technology, 2019,68(3): 2906-2920.
[34]	KANG J W , XIONG Z H , NIYATO D ,et al. Incentive mechanism for reliable federated learning:a joint optimization approach to combining reputation and contract theory[J]. IEEE Internet of Things Journal, 2019,6(6): 10700-10714.
[35]	KANG J W , XIONG Z H , NIYATO D ,et al. Reliable federated learning for mobile networks[J]. IEEE Wireless Communications, 2020,27(2): 72-80.
[36]	CATALANO D , FIORE D . Using linearly-homomorphic encryption to evaluate degree-2 functions on encrypted data[C]// Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015: 1518-1529.
[37]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.

Metrics

Recommended 0

No Suggested Reading articles found!

方案	保护模型参数	保护用户可靠性	保护聚合参数	用户退出鲁棒性	模型收敛效率
ecProbe^[17]	√	×	×	×	×
SAHPP^[19]	√	√	√	√	×
PPFDL^[21]	√	√	√	√	×
PPRFL	√	√	√	√	√
注：“√”表示满足对应功能，“×”表示不满足。

方案	用户与第三方	用户与服务器S₀	服务器S₀与S₁
SAHPP^[19]	O(n)	O(n²+nm)	—
PPFDL^[21]	—	O (nm)	O (κ(n+m))
PPRFL	—	O (nm)	O (m)

方案	采用STT前	采用STT后
SAHPP^[19]	O (nm)
PPFDL^[21]	O (2κnm)	O (2κ(n+m))
PPRFL	O (n+nm)	O (n+m)

协议	T_OT	T_GC	CMult	Mult	Enc	Dec
SecDiv^[21]	1	1	3	—	4	2
本文协议	—	—	3	1	3	2

Privacy-preserving federated learning framework with irregular-majority users

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 37

Related Articles 11

Metrics

Recommended 0

[1]	Feng YU, Qingxin LIN, Hui LIN, Xiaoding WANG. Privacy-enhanced federated learning scheme based on generative adversarial networks [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 113-122.
[2]	Jinyin CHEN, Rongchang LI, Guohan HUANG, Tao LIU, Haibin ZHENG, Yao CHENG. Survey on vertical federated learning: algorithm, privacy and security [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 1-20.
[3]	Zuobin YING, Yichen FANG, Yiwen ZHANG. Privacy-preserving federated learning framework with dynamic weight aggregation [J]. Chinese Journal of Network and Information Security, 2022, 8(5): 56-65.
[4]	Ming YANG, Xuexian HU, Qihui ZHANG, Jianghong WEI, Wenfen LIU. Federated learning scheme for mobile network based on reputation evaluation mechanism and blockchain [J]. Chinese Journal of Network and Information Security, 2021, 7(6): 99-112.
[5]	Fuyuan SONG, Zheng QIN, Jixin ZHANG, Yu LIU. Efficient and secure multi-user outsourced image retrieval scheme with access control [J]. Chinese Journal of Network and Information Security, 2021, 7(5): 29-39.
[6]	Chuanxin ZHOU, Yi SUN, Degang WANG, Huawei GE. Survey of federated learning research [J]. Chinese Journal of Network and Information Security, 2021, 7(5): 77-92.
[7]	Xinyu ZHANG, Bingsheng ZHANG, Quanrun MENG, Kui REN. Study on privacy preserving encrypted traffic detection [J]. Chinese Journal of Network and Information Security, 2021, 7(4): 101-113.
[8]	Ying WU,Xuan LI,Biao JIN,Rongrong JIN. Survey on the privacy-preserving content based image retrieval [J]. Chinese Journal of Network and Information Security, 2019, 5(4): 14-28.
[9]	Zijian BAO,Qinghao WANG,Yongxin ZHANG,Bin WANG,Ning LU,Wenbo SHI. Regulatory Bitcoin privacy-preserving mixing service [J]. Chinese Journal of Network and Information Security, 2019, 5(4): 40-51.
[10]	Jian-qiong HE,You-liang TIAN,Kai ZHOU. Provably secure social network privacy-preserving scheme [J]. Chinese Journal of Network and Information Security, 2016, 2(8): 62-67.
[11]	Ming-jie MA,Yue-jin DU,Feng-hua LI,IUJia-wen L. Review of semantic-based privacy-preserving approaches in LBS [J]. Chinese Journal of Network and Information Security, 2016, 2(12): 1-11.