MapReduce框架下支持差分隐私保护的随机梯度下降算法

doi:10.11959/j.issn.1000-436x.2018013

Abstract

Abstract:

Aiming at the contradiction between the efficiency and privacy of stochastic gradient descent algorithm in distributed computing environment,a stochastic gradient descent algorithm preserving differential privacy based on MapReduce was proposed.Based on the computing framework of MapReduce,the data were allocated randomly to each Map node and the Map tasks were started independently to execute the stochastic gradient descent algorithm.The Reduce tasks were appointed to update the model when the sub-target update models were meeting the update requirements,and to add Laplace random noise to achieve differential privacy protection.Based on the combinatorial features of differential privacy,the results of the algorithm is proved to be able to fulfill ε-differentially private.The experimental results show that the algorithm has obvious efficiency advantage and good data availability.

Key words: machine learning, stochastic gradient descent, MapReduce, differential privacy preserving, Laplace mechanism

CLC Number:

TP301

Yihan YU,Yu FU,Xiaoping WU. Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework[J]. Journal on Communications, 2018, 39(1): 70-77.

Figures/Tables 7

References 15

[1]	WU F , LI F G , KUMAR A ,et al. Bolt-on differential privacy for scalable stochastic gradient descent-based analytics[C]// The 2017 ACM International Conference on Management of Data. 2017: 1307-1322.
[2]	ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// The 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016: 308-318.
[3]	ZHAO P , ZHANG T . Stochastic optimization with importance sampling[J]. Eprint Arxiv, 2015: 1-9.
[4]	SCHMIDT M , ROUX N L , BACH F . Erratum to:minimizing finite sums with the stochastic average gradient[J]. Mathematical Programming, 2016,162(5):1.
[5]	MU Y , LIU W , LIU X ,et al. Stochastic gradient made stable:a manifold propagation approach for large-scale optimization[J]. IEEE Transactions on Knowledge ＆ Data Engineering, 2015,29(2): 458-471.
[6]	ZINKEVICH M , WEIMER M , SMOLA A J ,et al. Parallelized stochastic gradient descent[C]// The Conference on Neural Information Processing Systems. 2011: 2595-2603.
[7]	陈振宏, 兰艳艳, 郭嘉丰 ,等. 基于差异合并的分布式随机梯度下降算法[J]. 计算机学报, 2015,38(10): 2054-2063.
	CHEN Z H , LAN Y Y , GUO J F ,et al. Distributed stochastic gradient descent with discriminative aggregating[J]. Chinese Journal of Computers, 2015,38(10): 2054-2063.
[8]	ZHAO H , CANNY J F . Communication-efficient distributed stochastic gradient descent with butterfly mixing[D]. Berkeley,USA:University of California, 2012.
[9]	SONG S , CHAUDHURI K , SARWATE A D . Stochastic gradient descent with differentially private updates[C]// Global conference on Signal and Information Processing (GlobalSIP). 2013: 245-248.
[10]	BASSILY R , THAKURTA A . Private empirical risk minimization:Efficient algorithms and tight error bounds[C]// 2014 IEEE 55th Annual Symposium on Foundations of Computer Science (FOCS). 2014: 464-473.
[11]	DWORK C , MCSHERRY F , NISSIM K . Calibrating noise to sensitivity in private data analysis[J]. The VLDB Endowment, 2006,7(8): 637-648.
[12]	DWORK C , ROTH A . The Algorithmic foundations of differential privacy[M]. Now Publishers Inc, 2014.
[13]	CHAUDHURI K , MONTELEONI C , SARWATE A D . Differentially private empirical risk minimization[J]. Journal of Machine Learning Research, 2009,12(2): 1069-1109.
[14]	何贤芒, 王晓阳, 陈华辉 ,等. 差分隐私保护参数ε的选取研究[J]. 通信学报, 2015,36(12): 124-130.
	HE X M , WANG X Y , CHEN H H ,et al. Study on choosing the parameter ε in differential privacy[J]. Journal on Communications, 2015,36(12): 124-130.
[15]	MCSHERRY F D . Privacy integrated queries:an extensible platform for privacy-preserving data analysis[J]. Communication of the ACM, 2010,53(9): 89-97.

Metrics

Recommended 0

No Suggested Reading articles found!

类别	变量	说明
	N	数据记录总数
	ε	隐私预算量
	K _i、K_max	、Per、Cou 计数初值、计数最大值、数据利用率、计数阈值
	u	Map任务中模型更新次数
	U	Reduce任务中更新次数
数值型参数	μ	更新次数中间值
	E	阶段性误差标准
	Δerror	误差变化量
	Jud	测试数据判断值
	Error	最终模型误差值
	E_final	最终标准误差
	L (u)	第u 次更新时的更新次数阈值
函数	W (u)	第u次更新时分目标模型
	error (u)	第u次更新时W(u)相对误差计算数据集的误差
	n_i	第i条记录
一般符号	W_update	Reduce任务更新模型
	W_F	最终模型

Stochastic gradient descent algorithm preserving differential privacy in MapReduce framework

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 15

Related Articles 15

Metrics

Recommended 0

[1]	Qianyi DAI, Bin ZHANG, Song GUO, Kaiyong XU. Blockchain network layer anomaly traffic detection method based on multiple classifier integration [J]. Journal on Communications, 2023, 44(3): 66-80.
[2]	Gaofeng HE, Qianfeng WEI, Xiancai XIAO, Haiting ZHU, Bingfeng XU. Confirmation method for the detection of malicious encrypted traffic with data privacy protection [J]. Journal on Communications, 2022, 43(2): 156-170.
[3]	Zhibin FENG, Yuhua XU, Zhiyong DU, Xin LIU, Wen LI, Hao HAN, Xiaobo ZHANG. Active defense technology against intelligent jammer [J]. Journal on Communications, 2022, 43(10): 42-54.
[4]	Yanhui LU, Han LIU, Hang LI, Guangxu ZHU. Time series generation model based on multi-discriminator generative adversarial network [J]. Journal on Communications, 2022, 43(10): 167-176.
[5]	Kai MEI, Haitao ZHAO, Xiaoran LIU, Jun LIU, Jun XIONG, Baoquan REN, Jibo WEI. Efficient model-and-data based channel estimation algorithm [J]. Journal on Communications, 2022, 43(1): 59-70.
[6]	Changgen PENG, Ting GAO, Huilan LIU, Hongfa DING. PCA-based membership inference attack for machine learning models [J]. Journal on Communications, 2022, 43(1): 149-160.
[7]	Futai ZOU, Yue TAN, Lin WANG, Yongkang JIANG. Botnet detection based on generative adversarial network [J]. Journal on Communications, 2021, 42(7): 95-106.
[8]	Liu LIU, Jianhua ZHANG, Yuanyuan FAN, Li YU, Jiachi ZHANG. Survey of application of machine learning in wireless channel modeling [J]. Journal on Communications, 2021, 42(2): 134-153.
[9]	Yusun FU,Genke YANG. Application of artificial intelligence in mobile communication:challenge and practice [J]. Journal on Communications, 2020, 41(9): 190-201.
[10]	Tieming CHEN,Chengqiang JIN,Mingqi LYU,Tiantian ZHU. Intelligent detection method on network malicious traffic based on sample enhancement [J]. Journal on Communications, 2020, 41(6): 128-138.
[11]	Chunyu HAN,Yongzheng ZHANG,Yu ZHANG. Fast-flucos:malicious domain name detection method for Fast-flux based on DNS traffic [J]. Journal on Communications, 2020, 41(5): 37-47.
[12]	Xin ZHOU,Xiaoxin HE,Changwen ZHENG. Radio signal recognition based on image deep learning [J]. Journal on Communications, 2019, 40(7): 114-125.
[13]	Xuehui DU,Yangdong LIN,Yi SUN. Malicious PDF document detection based on mixed feature [J]. Journal on Communications, 2019, 40(2): 118-128.
[14]	Hongyu SUN,Yuan HE,Jice WANG,Ying DONG,Lipeng ZHU,He WANG,Yuqing ZHANG. Application of artificial intelligence technology in the field of security vulnerability [J]. Journal on Communications, 2018, 39(8): 1-17.
[15]	Yangchen HUANG,Yan JIA,Liang GAN,Jing XU,Jiuming HUANG,Zhonghe HE. Multi-factor person entity relation extraction model based on distant supervision [J]. Journal on Communications, 2018, 39(7): 103-112.