基于本地化差分隐私的联邦学习方法研究

doi:10.11959/j.issn.1000-436x.2022189

Abstract

Abstract:

As a type of collaborative machine learning framework, federated learning is capable of preserving private data from participants while training the data into useful models.Nevertheless, from a viewpoint of information theory, it is still vulnerable for a curious server to infer private information from the shared models uploaded by participants.To solve the inference attack problem in federated learning training, a local differential privacy federated learning (LDP-FL) approach was proposed.Firstly, to ensure the federated model training process was protected from inference attacks, a local differential privacy mechanism was designed for transmission of parameters in federated learning.Secondly, a performance loss constraint mechanism for federated learning was proposed and designed to reduce the performance loss of local differential privacy federated model by optimizing the constraint range of the loss function.Finally, the effectiveness of proposed LDP-FL approach was verified by comparative experiments on MNIST and Fashion MNIST datasets.

Key words: differential privacy, federated learning, deep learning

CLC Number:

TP309.2

Haiyan KANG, Yuanrui JI. Research on federated learning approach based on local differential privacy[J]. Journal on Communications, 2022, 43(10): 94-105.

Figures/Tables 7

符号	含义
N	联邦学习参与方数量
C_i	联邦学习中第i个参与方
D	所有参与方的训练数据集之和
D_i	第i个参与方对应的数据集
L_i	第i个参与方的本地损失函数
ε	本地化差分隐私定义中的隐私预算
δ	本地化差分隐私定义中相关参数
$M$	本地化差分隐私扰动机制
σ	本地化差分隐私噪声机制方差
q	联邦学习过程中每次迭代时的采样率
w	联邦学习中的模型参数
T	联邦学习总交流轮次
Δs	本地化差分隐私敏感度
p_i	第i个参与方的性能损失函数
P	本地化差分隐私整体性能损失
α(λ)	λ时刻的性能损失函数

References 19

[1]	YANG Q , LIU Y , CHEN T J ,et al. Federated machine learning:concept and applications[J]. ACM Transactions on Intelligent Systems and Technology, 2019,10(2): 1-19.
[2]	KAIROUZ P , MCMAHAN H B , AVENT B ,et al. Advances and open problems in federated learning[J]. Foundations and Trends in Machine Learning, 2021,14(1-2): 1-210.
[3]	方晨, 郭渊博, 王一丰 ,等. 基于区块链和联邦学习的边缘计算隐私保护方法[J]. 通信学报, 2021,42(11): 28-40.
	FANG C , GUO Y B , WANG Y F ,et al. Edge computing privacy protection method based on blockchain and federated learning[J]. Journal on Communications, 2021,42(11): 28-40.
[4]	莫梓嘉, 高志鹏, 杨杨 ,等. 面向车联网数据隐私保护的高效分布式模型共享策略[J]. 通信学报, 2022,43(4): 83-94.
	MO Z J , GAO Z P , YANG Y ,et al. Efficient distributed model sharing strategy for data privacy protection in Internet of vehicles[J]. Journal on Communications, 2022,43(4): 83-94.
[5]	刘艺璇, 陈红, 刘宇涵 ,等. 联邦学习中的隐私保护技术[J]. 软件学报, 2022,33(3): 1057-1092.
	LIU Y X , CHEN H , LIU Y H ,et al. Privacy-preserving techniques in federated learning[J]. Journal of Software, 2022,33(3): 1057-1092.
[6]	SONG M K , WANG Z B , ZHANG Z F ,et al. Analyzing user-level privacy attack against federated learning[J]. IEEE Journal on Selected Areas in Communications, 2020,38(10): 2430-2444.
[7]	LIU X Y , LI H W , XU G W ,et al. Privacy-enhanced federated learning against poisoning adversaries[J]. IEEE Transactions on Information Forensics and Security, 2021,16: 4574-4588.
[8]	PHONG L T , AONO Y , HAYASHI T ,et al. Privacy-preserving deep learning via additively homomorphic encryption[C]// Proceedings of IEEE Transactions on Information Forensics and Security. Piscataway:IEEE Press, 2019: 1333-1345.
[9]	OU W , ZENG J , GUO Z ,et al. A homomor-phic-encryption-based vertical federated learning scheme for rick management[J]. Computer Science and Information Systems, 2020,17(3): 819-834.
[10]	ZHU H F , MONG G R S , NG W K . Privacy-preserving weighted federated learning within the secret sharing framework[J]. IEEE Access, 2020,8: 198275-198284.
[11]	DWORK C , . Differential privacy[C]// Proceedings of 2006 International Colloquium on Automata,Languages and Programming (ICALP). Berlin:Springer, 2006: 1-12.
[12]	GEYER R C , KLEIN T , NABI M . Differentially private federated learning:a client level perspective[J]. arXiv Preprint,arXiv:1712.07557, 2017.
[13]	ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2016: 308-318.
[14]	ZHAO C X , SVN Y , WANG D G . Federated learning with Gaussian differential privacy[C]// Proceedings of the 2020 2nd International Conference on Robotics,Intelligent Control and Artificial Intelligence. Piscataway:IEEE Press, 2020: 296-301.
[15]	WEI K , LI J , DING M ,et al. Federated learning with differential privacy:algorithms and performance analysis[J]. IEEE Transactions on Information Forensics and Security, 2020,15: 3454-3469.
[16]	TRUEX S , LIU L , CHOW K H ,et al. LDP-Fed:federated learning with local differential privacy[C]// Proceedings of the Third ACM International Workshop on Edge Systems,Analytics and Networking. New York:ACM Press, 2020: 61-66.
[17]	LIU R X , CAO Y , YOSHIKAWA M ,et al. FedSel:federated SGD under local differential privacy with top-k dimension selection[C]// International Conference on Database Systems for Advanced Applications. Berlin:Springer, 2020: 485-501.
[18]	ZHAO Y , ZHAO J , YANG M M ,et al. Local differential privacy-based federated learning for Internet of things[J]. IEEE Internet of Things Journal, 2021,8(11): 8836-8853.
[19]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[J]. arXiv Preprint,arXiv:1602.05629, 2016.

Metrics

Recommended 0

No Suggested Reading articles found!

分类	方案	概述	不足
加密	文献[7]	基于同态加密思想设计联邦学习参数保护方法	计算代价高
	文献[8]	由终端自行生成密钥，联邦模型训练时结合加法同态技术对参数进行保护	计算代价高
	文献[9]	引入第三方进行私钥分配后再使用同态加密技术	计算代价高
	文献[10]	结合秘密分享技术对参与方传递的参数进行保护	通信开销大
中心化扰动	文献[12]	设计用户级别的差分隐私联邦学习算法	依赖中心参数服务器
	文献[14]	利用高斯噪声机制对参数进行保护	依赖中心参数服务器
	文献[15]	设计分阶段差分隐私联邦学习模型NbAFL	依赖中心参数服务器
本地化扰动	文献[16]	利用指数机制思想设计差分隐私参数扰动	性能损失较大
	文献[17]	根据权重对梯度进行选择后再进行扰动	性能损失较大
	文献[18]	选择部分梯度后将结果进行离散后再添加噪声	性能损失较大

数据集	方法	迭代轮次/次
数据集	方法	25	50	75	100	125	150
	LDP-FL	1.87	1.84	1.82	1.72	1.75	1.72
MNIST	NbAFL	1.89	1.88	1.90	1.81	1.82	1.79
	CL-FL	1.92	1.90	1.99	1.84	1.88	1.85
	LDP-FL	2.21	2.14	2.13	2.20	2.17	2.15
Fashion MINST	NbAFL	2.33	2.24	2.15	2.44	2.35	2.46
	CL-FL	2.52	2.44	2.47	2.58	2.68	2.77

Research on federated learning approach based on local differential privacy

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 19

Related Articles 15

Metrics

Recommended 0

[1]	Dongyu CHEN, Hua CHEN, Limin FAN, Yifang FU, Jian WANG. Research on test strategy for randomness based on deep learning [J]. Journal on Communications, 2023, 44(6): 23-33.
[2]	Rongpeng LI, Bingyan WANG, Honggang ZHANG, Zhifeng ZHAO. Design of knowledge enhanced semantic communication receiver [J]. Journal on Communications, 2023, 44(6): 70-76.
[3]	Xindi MA, Qinghua LI, Qi JIANG, Zhuo MA, Sheng GAO, Youliang TIAN, Jianfeng MA. Byzantine-robust federated learning over Non-IID data [J]. Journal on Communications, 2023, 44(6): 138-153.
[4]	Shuai MA, Ke PEI, Huayan QI, Hang LI, Wen CAO, Hongmei WANG, Hailiang XIONG, Shiyin LI. Research on geomagnetic indoor high-precision positioning algorithm based on generative model [J]. Journal on Communications, 2023, 44(6): 211-222.
[5]	Kaiju LI, Qiang XU, Hao WANG. Communication-efficient federated learning method via redundant data elimination [J]. Journal on Communications, 2023, 44(5): 79-93.
[6]	Shengxing YU, Zekai CHEN, Zhong CHEN, Ximeng LIU. DAGUARD: distributed backdoor attack defense scheme under federated learning [J]. Journal on Communications, 2023, 44(5): 110-122.
[7]	Hui JIANG, Tianliu HE, Min LIU, Sheng SUN, Yuwei WANG. High-performance federated continual learning algorithm for heterogeneous streaming data [J]. Journal on Communications, 2023, 44(5): 123-136.
[8]	Youliang TIAN, Shihong WU, Ta LI, Lindong WANG, Hua ZHOU. Federated learning optimization algorithm based on incentive mechanism [J]. Journal on Communications, 2023, 44(5): 169-180.
[9]	Jiale ZHANG, Chengcheng ZHU, Xiaobing SUN, Bing CHEN. Membership inference attack and defense method in federated learning based on GAN [J]. Journal on Communications, 2023, 44(5): 193-205.
[10]	Tao FENG, Liqiu CHEN, Junli FANG, Jianming SHI. Blockchain data sharing scheme based on localized difference privacy and attribute-based searchable encryption [J]. Journal on Communications, 2023, 44(5): 224-233.
[11]	Shufen ZHANG, Yanling DONG, Jingcheng XU, Haoshi WANG. AdaBoost algorithm based on target perturbation [J]. Journal on Communications, 2023, 44(2): 198-209.
[12]	Shengxing YU, Zhong CHEN. Efficient secure federated learning aggregation framework based on homomorphic encryption [J]. Journal on Communications, 2023, 44(1): 14-28.
[13]	Lingtao TANG, Di WANG, Shengyun LIU. Data augmentation scheme for federated learning with non-IID data [J]. Journal on Communications, 2023, 44(1): 164-176.
[14]	Chengsheng YUAN, Qiang GUO, Zhangjie FU. Copyright protection algorithm based on differential privacy deep fake fingerprint detection model [J]. Journal on Communications, 2022, 43(9): 181-193.
[15]	Hanyi WANG, Xiaoguang LI, Wenqing BI, Yahong CHEN, Fenghua LI, Ben NIU. Multi-level local differential privacy algorithm recommendation framework [J]. Journal on Communications, 2022, 43(8): 52-64.