差分隐私GAN梯度裁剪阈值的自适应选取方法

doi:10.11959/j.issn.2096-109x.2018041

网络与信息安全学报 ›› 2018, Vol. 4 ›› Issue (5): 10-20.doi: 10.11959/j.issn.2096-109x.2018041

差分隐私GAN梯度裁剪阈值的自适应选取方法

郭鹏^1,²,钟尚平^1,²,陈开志^1,²,程航^1,²

¹ 福州大学数学与计算机科学学院，福建福州 350116
² 网络系统信息安全福建省高校重点实验室，福建福州 350116

修回日期:2018-04-02 出版日期:2018-05-15 发布日期:2018-08-04
作者简介:郭鹏（1991-），男，河南郸城人，福州大学硕士生，主要研究方向为信息安全、机器学习。|钟尚平（1969-），男，福建武平人，福州大学教授，主要研究方向为信息安全，机器学习、智能图像分析。|陈开志（1983-），男，福建福州人，福州大学讲师，主要研究方向为信息安全、图像安全。|程航（1979-），男，福建连江人，福州大学副教授，主要研究方向为多媒体安全、加密域图像检索、图像处理。
基金资助:
福建省自然科学基金资助项目(2017J01502);福州大学博士科研基金资助项目(XRC-17015)

Adaptive selection method of differential privacy GAN gradient clipping thresholds

Peng GUO^1,²,Shangping ZHONG^1,²,Kaizhi CHEN^1,²,Hang CHENG^1,²

¹ College of Mathematics and Computer Science,Fuzhou University,Fuzhou 350116,China
² Network System Information Security Fujian Provincial University Key Laboratory,Fuzhou 350116,China

Revised:2018-04-02 Online:2018-05-15 Published:2018-08-04
Supported by:
The Natural Science Foundation of Fujian Province(2017J01502);The Scientific Research Foundation of Fuzhou University(XRC-17015)

摘要/Abstract

摘要：

提出自适应选取差分隐私 GAN 梯度裁剪阈值的方法。该方法假设可以接触到与隐私数据同分布的小部分公开数据，通过从公开数据中随机选取一批数据，设置裁剪阈值为这批数据的平均梯度范数，迭代上述操作直到网络聚合。在 Mnist 和 Cifar10 数据集上对所提方法进行了实验验证，结果表明，在合理隐私预算下与差分隐私辅助分类GAN相比，卷积神经网络（CNN）分类器准确率提高1%~4%，而评估分数（inception scores）提升0.6~1.2。

关键词: 生成对抗网络, 差分隐私保护, 梯度裁剪阈值, 自适应选取

Abstract:

A method of adaptive selection of differential privacy GAN gradient clipping threshold was proposed.The method assumes that a small portion of public data that is identically distributed with the private data can be accessed,a batch of data is randomly selected from the public data,a clipping threshold is set as an average gradient norm of the batch of data,and the above operations are iterated until the network converges.The method was verified on the Mnist and Cifar10 data sets.The results show that under a reasonable privacy budget,the accuracy of CNN classifiers is improved by 1%~4% compared with the differential privacy auxiliary classifier GAN,inception scores increased by 0.6~1.2.

Key words: GAN, differential privacy protection, gradient clipping thresholds, adaptive selection

中图分类号:

TP309

郭鹏, 钟尚平, 陈开志, 程航. 差分隐私GAN梯度裁剪阈值的自适应选取方法[J]. 网络与信息安全学报, 2018, 4(5): 10-20.

Peng GUO, Shangping ZHONG, Kaizhi CHEN, Hang CHENG. Adaptive selection method of differential privacy GAN gradient clipping thresholds[J]. Chinese Journal of Network and Information Security, 2018, 4(5): 10-20.

图/表 9

图1

图2

图3

图4

图5

图6

图7

图8

表1

参考文献 18

[1]	SHOKRI R , SHMATIKOV V . Privacy-preserving deep learning[C]// The 53rd IEEE Annual Allerton Conference on Communication,Control,and Computing. 2015: 909-910.
[2]	ZHU T , LI G , ZHOU W ,et al. Differentially private data publishing and analysis:a survey[J]. IEEE Transactions on Knowledge ＆ Data Engineering, 2017,29(8): 1619-1638.
[3]	MOHASSEL P , ZHANG Y . SecureML:a system for scalable privacy-preserving machine learning[C]// The 38th IEEE Symposium on Security and Privacy. 2017: 19-38.
[4]	SWEENEY L . K-Anonymity:a model for protecting privacy[J]. International Journal of UncertainTy Fuzziness and Knowledge Based Systems, 2002,10(5): 557-570.
[5]	MACHANAVAJJHALA A , GEHRKE J , KIFER D ,et al. l-diversity:privacy beyond k-anonymity[C]// The 22nd IEEE International Conference on Data Engineering.Atlanta Georgia. 2006: 24-24.
[6]	LI N , LI T , VENKATASUBRAMANIAN S ,et al. T-closeness:privacy beyond k-anonymity and l-diversity[C]// The 23rd IEEE International Conference on Data Engineering. 2007: 106-115.
[7]	DWORK C . Differential privacy[M]// Encyclopedia of Cryptography and Security. Springer, 2011: 338-340.
[8]	ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// The ACM SIGSAC Conference on Computer and Communications Security. 2016: 308-318.
[9]	PAPERNOT N , ABADI M , ERLINGSSON U ,et al. Semi-supervised knowledge transfer for deep learning from private training data[C]// The 5th International Conference on Learning Representations. 2017
[10]	ACS G , MELIS L , CASTELLUCCIA C ,et al. Differentially private mixture of generative neural networks[C]// The 17th IEEE International Conference on Data Mining. 2017: 715-720.
[11]	BEAULIEU-JONES B K , WU Z S , WILLIAMS C ,et al. Privacy-preserving generative deep neural networks support clinical data sharing[R]. 2017.
[12]	ZHANG X , JI S , WANG T ,et al. Differentially private releasing via Deep Generative Model[EB/OL]. . 2018.
[13]	DWORK C , . Differential privacy:a survey of results[C]// International Conference on Theory and Applications of Models of Computation.Springer,Berlin,Heidelberg. 2008: 1-19.
[14]	DWORK C , ROTH A . The algorithmic foundations of differential privacy[J]. Foundations and Trends in Theoretical Computer Science, 2014,9(3/4): 211-407.
[15]	GOODFELLOW I , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial nets[C]// Advances in Neural Information Processing Systems. 2014: 2672-2680.
[16]	ODENA A , OLAH C , SHLENS J . Conditional image synthesis with auxiliary classifier GANs[C]// International Conference on Machine Learning. 2017: 2642-2651.
[17]	KAIROUZ P , OH S , VISWANATH P ,et al. The composition theorem for differential privacy[J]. IEEE Transactions on Information Theory, 2013,63(6): 4037-4049.
[18]	SALIMANS T , GOODFELLOW I , ZAREMBA W ,et al. Improved techniques for training GANs[C]// Advances in Neural Information Processing Systems. 2016: 2234-2242.

数据集	类型	数量	(ε,δ)	分数
	真实数据	60 000	—	9.96±0.03
Mnist	ACGAN生成数据	60 000	—	6.59±0.03
	DPACGAN生成数据	60 000	(4,10^?5)	9.0±0.03
	本文方法生成数据	60 000	(4,10^?5)	7.76±0.03
	真实数据	50 000	—	11.24±0.12
Cifar10	ACGAN生成数据	50 000	—	8.25±0.07
	DPACGAN生成数据	50 000	(8,10^?5)	6.68±0.06
	本文方法生成数据	50 000	(8,10^?5)	7.25±0.07

差分隐私GAN梯度裁剪阈值的自适应选取方法

Adaptive selection method of differential privacy GAN gradient clipping thresholds

在线阅读

pdf下载

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 18

相关文章 4

Metrics

推荐阅读 0

[1]	余锋, 林庆新, 林晖, 汪晓丁. 基于生成对抗网络的隐私增强联邦学习方案[J]. 网络与信息安全学报, 2023, 9(3): 113-122.
[2]	陈万泽, 黄丽清, 陈家祯, 叶锋, 黄添强, 罗海峰. 融合小波快捷连接生成对抗网络的面部性别伪造[J]. 网络与信息安全学报, 2023, 9(3): 150-160.
[3]	王正龙, 张保稳. 生成对抗网络研究综述[J]. 网络与信息安全学报, 2021, 7(4): 68-85.
[4]	张煜,吕锡香,邹宇聪,李一戈. 基于生成对抗网络的文本序列数据集脱敏[J]. 网络与信息安全学报, 2020, 6(4): 109-119.