基于生成对抗网络的隐私增强联邦学习方案

doi:10.11959/j.issn.2096-109x.2023043

网络与信息安全学报 ›› 2023, Vol. 9 ›› Issue (3): 113-122.doi: 10.11959/j.issn.2096-109x.2023043

基于生成对抗网络的隐私增强联邦学习方案

余锋¹^,², 林庆新³, 林晖¹^,², 汪晓丁¹^,²

¹ 福建师范大学计算机与网络空间安全学院，福建福州 350017
² 网络安全与教育信息化福建省高校工程研究中心（福建师范大学），福建福州 350017
³ 福州大学至诚学院，福建福州 350002

修回日期:2022-08-30 出版日期:2023-06-25 发布日期:2023-06-01
作者简介:余锋（1998– ），男，湖北大悟人，福建师范大学硕士生，主要研究方向为联邦学习、隐私保护
林庆新（1979- ），男，福建安溪人，福州大学副教授，主要研究方向为计算机网络安全
林晖（1977– ），男，福建福州人，博士，福建师范大学教授、博士生导师，主要研究方向为机器学习、移动边缘计算和无线网络信息安全
汪晓丁（1982- ），男，福建福州人，福建师范大学副教授，主要研究方向为网络优化与无线通信网络
基金资助:
国家自然科学基金(U1905211);国家自然科学基金(61702103);福建省自然科学基金(2020J01167);福建省自然科学基金(2020J01169)

Privacy-enhanced federated learning scheme based on generative adversarial networks

Feng YU¹^,², Qingxin LIN³, Hui LIN¹^,², Xiaoding WANG¹^,²

¹ College of Computer and Cyber Security, Fujian Normal University, Fuzhou 350017, China
² Engineering Research Center of Cyber Security and Education Informatization, Fujian Province University, Fuzhou 350117, China
³ Zhicheng College, Fuzhou University, Fuzhou 350002, China

Revised:2022-08-30 Online:2023-06-25 Published:2023-06-01
Supported by:
The National Natural Science Foundation of China(U1905211);The National Natural Science Foundation of China(61702103);The Natural Science Foundation of Fujian Province(2020J01167);The Natural Science Foundation of Fujian Province(2020J01169)

摘要/Abstract

摘要：

联邦学习作为一种分布式机器学习范式，其具有隐私保护能力和异构协作等特性，引起了研究者极大的关注。然而，研究工作表明通过梯度可以确定一个确切的数据记录或一个具有特定属性的数据记录是否包含在其他参与者的批处理中，甚至揭露参与者的训练数据，通常称之为“梯度泄露”。同时，当前隐私增强联邦学习方法的工作可能存在准确率下降或者计算通信开销增加等问题，甚至引发新的不安全因素。因此，提出一种差分隐私增强的生成对抗网络模型，该模型向vanilla GAN中引入了识别器，通过生成器与鉴别器、生成器与识别器两个博弈过程，生成器合成的数据尽可能接近输入数据的同时满足差分隐私的约束。将此模型应用到联邦学习框架中，在一定程度上保证了模型准确率，并且提高了联邦学习框架的隐私保护能力。仿真实验验证了所提方案在客户端/服务器联邦学习架构下的有效性，相比 DP-SGD 方法，所提方案平衡了数据隐私性与实用性而不是以牺牲准确率为代价来增强隐私保护能力。从理论上分析了所提模型在点对点（P2P，peer-to-peer）架构下的可用性，并讨论了未来研究工作。

关键词: 联邦学习, 梯度泄露, 隐私增强, 生成对抗网络, 差分隐私

Abstract:

Federated learning, a distributed machine learning paradigm, has gained a lot of attention due to its inherent privacy protection capability and heterogeneous collaboration.However, recent studies have revealed a potential privacy risk known as “gradient leakage”, where the gradients can be used to determine whether a data record with a specific property is included in another participant’s batch, thereby exposing the participant’s training data.Current privacy-enhanced federated learning methods may have drawbacks such as reduced accuracy, computational overhead, or new insecurity factors.To address this issue, a differential privacy-enhanced generative adversarial network model was proposed, which introduced an identifier into vanilla GAN, thus enabling the input data to be approached while satisfying differential privacy constraints.Then this model was applied to the federated learning framework, to improve the privacy protection capability without compromising model accuracy.The proposed method was verified through simulations under the client/server (C/S) federated learning architecture and was found to balance data privacy and practicality effectively compared with the DP-SGD method.Besides, the usability of the proposed model was theoretically analyzed under a peer-to-peer (P2P) architecture, and future research work was discussed.

Key words: federated learning, gradient leakage, privacy enhancement, generative adversarial network, differential privacy

中图分类号:

TP393

余锋, 林庆新, 林晖, 汪晓丁. 基于生成对抗网络的隐私增强联邦学习方案[J]. 网络与信息安全学报, 2023, 9(3): 113-122.

Feng YU, Qingxin LIN, Hui LIN, Xiaoding WANG. Privacy-enhanced federated learning scheme based on generative adversarial networks[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 113-122.

图/表 8

图1

图2

图3

图4

图5

图6

表1

表2

参考文献 26

[1]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[J]. arXiv Preprint arXiv:1602.05629, 2017.
[2]	ZHU L , LIU Z , HAN S . Deep leakage from gradients[C]// Advances in Neural Information Processing Systems. 2019:32.
[3]	ZHAO B , MOPURI KR , BILEN H . IDLG:improved deep leakage from gradients[J]. arXiv Preprint arXiv:2001.02610, 2020.
[4]	GEIPING J , BAUERMEISTER H , DR?GE H , ,et al. Inverting gradients:how easy is it to break privacy in federated learning[J]. arXiv Preprint arXiv:2003.14053, 2020.
[5]	ZHAO Y , ZHAO J , YANG M ,et al. Local differential privacy based federated learning for internet of things[J]. arXiv Preprint arXiv:2004.08856, 2020.
[6]	LI Y , ZHOU Y P , JOLFAEI A ,et al. Privacy-preserving federated learning framework based on chained secure multiparty computing[J]. IEEE Internet of Things Journal, 2021,8(8): 6178-6186.
[7]	TRUEX S , BARACALDO N , ANWAR A ,et al. A hybrid approach to privacy-preserving federated learning[J]. Informatik Spektrum, 2019,42(5): 356-357.
[8]	杨庚, 王周生 . 联邦学习中的隐私保护研究进展[J]. 南京邮电大学学报(自然科学版), 2020,40(5): 204-214.
	YANG G , WANG Z S . Survey on privacy preservation in federated learning[J]. Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition), 2020,40(5): 204-214.
[9]	陈兵, 成翔, 张佳乐 ,等. 联邦学习安全与隐私保护综述[J]. 南京航空航天大学学报, 2020,52(5): 675-684.
	CHEN B , CHENG X , ZHANG J L ,et al. Survey of security and privacy in federated learning[J]. Journal of Nanjing University of Aeronautics ＆ Astronautics, 2020,52(5): 675-684.
[10]	MOHASSEL P , ZHANG Y . SecureML:a system for scalable privacy-preserving machine learning[C]// 2017 IEEE Symposium on Security and Privacy (SP). 2017: 19-38.
[11]	KILBERTUS N , GASCON A , KUSNER M ,et al. Blind justice:fairness with encrypted sensitive attributes[C]// Proceedings of the 35th International Conference on Machine Learning. 2018: 2630-2639.
[12]	HITAJ B , ATENIESE G , PEREZ-CRUZ F . Deep models under the gan:information leakage from collaborative deep learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 603-618.
[13]	WANG Z B , SONG M K , ZHANG Z F ,et al. Beyond inferring class representatives:user-level privacy leakage from federated learning[C]// Proceedings of IEEE INFOCOM 2019-IEEE Conference on Computer Communications. 2019: 2512-2520.
[14]	HARDY C , LE MERRER E , SERICOLA B . MD-GAN:multi-discriminator generative adversarial networks for distributed datasets[C]// Proceedings of 2019 IEEE International Parallel and Distributed Processing Symposium. 2019: 866-877.
[15]	RASOULI M , SUN T , RAJAGOPAL R . FedGAN:federated generative adversarial networks for distributed data[J]. arXiv Preprint arXiv:2006.07228, 2020.
[16]	MUGUNTHAN V , GOKUL V , KAGAL L ,et al. Bias-free FedGAN:a federated approach to generate bias-free datasets[J]. arXiv Preprint arXiv:2103.09876, 2021.
[17]	ZHAO Z , BIRKE R , KUNAR A ,et al. Fed-TGAN:federated learning framework for synthesizing tabular data[J]. arXiv:2108.07927, 2021.
[18]	DWORK C , KENTHAPADI K , MCSHERRY F ,et al. Our data,ourselves:privacy via distributed noise generation[C]// Advances in Cryptology - Eurocrypt 2006. 2006: 486-503.
[19]	DWORK C , MCSHERRY F , NISSIM K ,et al. Calibrating noise to sensitivity in private data analysis[C]// Theory of Cryptography. 2006: 265-284.
[20]	GOODFELLOW I , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial networks[J]. Communications of the ACM, 2020,63(11): 139-144.
[21]	MELIS L , SONG C , DE CRISTOFARO E ,et al. Exploiting unintended feature leakage in collaborative learning[C]// 2019 IEEE Symposium on Security and Privacy (SP). 2019: 691-706.
[22]	ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016: 308-318.
[23]	AUGENSTEIN S , MCMAHAN H B , RAMAGE D ,et al. Generative models for effective ML on private,decentralized datasets[J]. arXiv:1911.06679,B2020.
[24]	SALIMANS T , GOODFELLOW I , ZAREMBA W ,et al. Improved techniques for training GANs[J]. arXiv Preprint arXiv:1606.03498, 2016.
[25]	HEUSEL M , RAMSAUER H , UNTERTHINER T ,et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[J]. arXiv Preprint arXiv:1706.08500, 2018.
[26]	NGUYEN D C , DING M , PHAM Q V ,et al. Federated learning meets blockchain in edge computing:opportunities and challenges[J]. IEEE Internet of Things Journal, 2021,8(16): 12806-12825.

模型	分类准确率
模型	No	GAN	DP-SGD	DP-enabled GAN
MLP	97%	82%	60%	78%
CNN	99%	82%	63%	79%

模型	分类准确率
模型	No	GAN	DP-SGD	DP-enabled GAN
MLP	86%	75%	50%	65%
CNN	89%	72%	45%	64%

基于生成对抗网络的隐私增强联邦学习方案

Privacy-enhanced federated learning scheme based on generative adversarial networks

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 26

相关文章 15

Metrics

推荐阅读 0

[1]	陈万泽, 黄丽清, 陈家祯, 叶锋, 黄添强, 罗海峰. 融合小波快捷连接生成对抗网络的面部性别伪造[J]. 网络与信息安全学报, 2023, 9(3): 150-160.
[2]	陈晋音, 李荣昌, 黄国瀚, 刘涛, 郑海斌, 程瑶. 纵向联邦学习方法及其隐私和安全综述[J]. 网络与信息安全学报, 2023, 9(2): 1-20.
[3]	应作斌, 方一晨, 张怡文. 动态聚合权重的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022, 8(5): 56-65.
[4]	陈前昕, 毕仁万, 林劼, 金彪, 熊金波. 支持多数不规则用户的隐私保护联邦学习框架[J]. 网络与信息安全学报, 2022, 8(1): 139-150.
[5]	杨明, 胡学先, 张启慧, 魏江宏, 刘文芬. 基于信誉评估机制和区块链的移动网络联邦学习方案[J]. 网络与信息安全学报, 2021, 7(6): 99-112.
[6]	周传鑫, 孙奕, 汪德刚, 葛桦玮. 联邦学习研究综述[J]. 网络与信息安全学报, 2021, 7(5): 77-92.
[7]	王正龙, 张保稳. 生成对抗网络研究综述[J]. 网络与信息安全学报, 2021, 7(4): 68-85.
[8]	张心语, 张秉晟, 孟泉润, 任奎. 隐私保护的加密流量检测研究[J]. 网络与信息安全学报, 2021, 7(4): 101-113.
[9]	赵明烽, Lei Chen, 钟洋, 熊金波. 移动边缘群智感知动态隐私度量模型与评价机制[J]. 网络与信息安全学报, 2021, 7(1): 157-166.
[10]	张煜,吕锡香,邹宇聪,李一戈. 基于生成对抗网络的文本序列数据集脱敏[J]. 网络与信息安全学报, 2020, 6(4): 109-119.
[11]	何贤芒. 基于差分隐私保护技术的多方求和查询方法[J]. 网络与信息安全学报, 2020, 6(3): 14-18.
[12]	杨磊,郑啸,赵伟. 基于差分隐私的非等距直方图发布方法[J]. 网络与信息安全学报, 2020, 6(3): 39-49.
[13]	刘慧,毕仁万,熊金波,赵明烽,金彪,林劼. 移动群智感知中基于雾节点协作的感知用户身份隐私保护[J]. 网络与信息安全学报, 2019, 5(6): 75-84.
[14]	李森有,季新生,游伟. 基于置信度分析的差分隐私保护参数配置方法研究[J]. 网络与信息安全学报, 2019, 5(4): 29-39.
[15]	郭鹏, 钟尚平, 陈开志, 程航. 差分隐私GAN梯度裁剪阈值的自适应选取方法[J]. 网络与信息安全学报, 2018, 4(5): 10-20.