物联网环境中基于深度学习的差分隐私预算优化方法

doi:10.11959/j.issn.2096-3750.2022.00264

摘要/Abstract

摘要：

为有效处理物联网大规模应用所带来的海量数据，深度学习在物联网环境中得到广泛应用。然而，深度模型在训练过程中，存在推理攻击、模型逆向攻击等安全威胁，这会导致输入模型中的原始数据泄露。应用差分隐私对深度模型训练过程的参数进行保护，是解决该问题的有效方式。基于此提出一种物联网环境中基于深度学习的差分隐私预算优化方法，根据参数迭代变化规律，自适应地分配不同预算；为避免噪声过大的问题，引入正则化项对扰动项进行约束，既防止神经网络过拟合，又有助于学习模型的显著特征。实验表明，所提方法可有效增强模型的泛化能力；随着模型迭代次数增加，加噪后训练得到的模型，与使用原始数据训练得到的模型，二者精度差值低于0.5%。因此，所提方法既可实现用户隐私保护，同时有效保证模型可用性，实现了隐私性和可用性的平衡。

关键词: 物联网, 差分隐私, 正则化, 深度学习, 隐私预算

Abstract:

In order to effectively process the massive data brought by the large-scale application of the internet of things (IoT), deep learning is widely used in IoT environment.However, in the training process of deep learning, there are security threats such as reasoning attacks and model reverse attacks, which can lead to the leakage of the original data input to the model.Applying differential privacy to protect the training process parameters of the deep model is an effective way to solve this problem.A differential privacy budget optimization method was proposed based on deep learning in IoT, which adaptively allocates different budgets according to the iterative change of parameters.In order to avoid the excessive noise, a regularization term was introduced to constrain the disturbance term.Preventing the neural network from over fitting also helps to learn the salient features of the model.Experiments show that this method can effectively enhance the generalization ability of the model.As the number of iterations increases, the accuracy of the model trained after adding noise is almost the same as that obtained by training using the original data, which not only achieves privacy protection, but also guarantees the availability, which means balance the privacy and availability.

Key words: IoT, differential privacy, regularization, deep learning, privacy budget

中图分类号:

TP391

罗丹, 徐茹枝, 关志涛. 物联网环境中基于深度学习的差分隐私预算优化方法[J]. 物联网学报, 2022, 6(2): 65-76.

Dan LUO, Ruzhi XU, Zhitao GUAN. Differential privacy budget optimization based on deep learning in IoT[J]. Chinese Journal on Internet of Things, 2022, 6(2): 65-76.

图/表 7

图1

图2

图3

图4

图5

图6

图7

参考文献 25

[1]	杨毅宇, 周威, 赵尚儒 ,等. 物联网安全研究综述：威胁、检测与防御[J]. 通信学报, 2021,42(8): 188-205.
	YANG Y Y , ZHOU W , ZHAO S R ,et al. Survey of IoT security research:threats,detection and defense[J]. Journal on Communications, 2021,42(8): 188-205.
[2]	吕建新, 郑伟, 马林 ,等. 基于词向量语义扩展的网络文本特征选择方法研究[J]. 情报科学, 2019,37(12): 47-51.
	LV J X , ZHENG W , MA L ,et al. Feature selection method of the network text based on semantic extension with word vector[J]. Information Science, 2019,37(12): 47-51.
[3]	孟仕林, 赵蕴龙, 关东海 ,等. 融合情感与语义信息的情感分析方法[J]. 计算机应用, 2019,39(7): 1931-1935.
	MENG S L , ZHAO Y L , GUAN D H ,et al. Sentiment analysis method combining sentiment and semantic information[J]. Journal of Computer Applications, 2019,39(7): 1931-1935.
[4]	LI T , LI J , CHEN X F ,et al. NPMML:a framework for non-interactive privacy-preserving multi-party machine learning[J]. IEEE Transactions on Dependable and Secure Computing, 2021,18(6): 2969-2982.
[5]	ZHANG X L , FU A M , WANG H Q ,et al. A privacy-preserving and verifiable federated learning scheme[C]// Proceedings of ICC 2020 2020 IEEE International Conference on Communications. Piscataway:IEEE Press, 2020: 1-6.
[6]	SUH J , TANAKA T . Encrypted value iteration and temporal difference learning over leveled homomorphic encryption[C]// Proceedings of 2021 American Control Conference (ACC). Piscataway:IEEE Press, 2021.
[7]	WANG Y C , LIANG X L , HEI X H ,et al. Deep learning data privacy protection based on homomorphic encryption in AIoT[J]. Mobile Information Systems,2021, 2021:5510857.
[8]	YE H , LIU J Q , WANG W ,et al. Secure and efficient outsourcing differential privacy data release scheme in Cyber-physical system[J]. Future Generation Computer Systems, 2020,108: 1314-1323.
[9]	BU Z Q , WANG H , LONG Q ,et al. On the convergence of deep learning with differential privacy[EB]. 2021.
[10]	BU Z Q , GOPI S , KULKARNI J ,et al. Fast and memory efficient differentially private-SGD via JL projections[EB]. 2021.
[11]	CHEN X , WU S Z , HONG M . Understanding gradient clipping in private SGD:A geometric perspective[J]. Advances in Neural Information Processing Systems, 2020,33: 13773-13782.
[12]	KOSKELA A , JALKO J , HONKELA A . Computing tight differential privacy guarantees using fft[C]// International Conference on Artificial Intelligence and Statistics. Online:PMLR, 2020: 2560-2569.
[13]	GHAZI B , GOLOWICH N , KUMAR R ,et al. On deep learning with label differential privacy[EB]. 2021.
[14]	YUAN S , SHEN M , MIRONOV I ,et al. Practical,label private deep learning training based on secure multiparty computation and differential privacy[EB]. 2021.
[15]	ABADI M , CHU A , GOODFELLOW I ,et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2016: 308-318.
[16]	DWORK C , . Differential privacy[C]// Proceedings of 33th International Colloquium on Automata,Languages and Programming. Berlin:Springer, 2006: 1-12.
[17]	DWORK C , . Differential privacy:a survey of results[C]// Proceedings of the 5th International Conference on Theory and Applications of Models of Computation. Berlin:Springer-Verlag, 2008: 1-19.
[18]	DWORK C , MCSHERRY F , NISSIM K ,et al. Calibrating noise to sensitivity in private data analysis[C]// Theory of Cryptography. Berlin,Heidelberg:Springer, 2006: 265-284.
[19]	ZINKEVICH M , WEIMER M , LI L ,et al. Parallelized stochastic gradient descent[C]// Proceedings of Advances in neural information processing systems. Vancouver,Canada:NIPS, 2010: 2595-2603.
[20]	CHANG D Q , LIN M , ZHANG C S . On the generalization ability of online gradient descent algorithm under the quadratic growth condition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(10): 5008-5019.
[21]	BUKOVSKY I , HOMMA N . An approach to stable gradient-descent adaptation of higher order neural units[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017,28(9): 2022-2034.
[22]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[23]	汪小寒, 韩慧慧, 张泽培 ,等. 树索引数据差分隐私预算分配方法[J]. 计算机应用, 2018,38(7): 1960-1966.
	WANG X H , HAN H H , ZHANG Z P ,et al. Differential privacy budget allocation method for data of tree index[J]. Journal of Computer Applications, 2018,38(7): 1960-1966.
[24]	GONG M G , FENG J L , XIE Y . Privacy-enhanced multi-party deep learning[J]. Neural Networks, 2020,121: 484-496.
[25]	王璇 . 差分隐私保护中隐私预算的优化与应用[D]. 南京:南京邮电大学, 2019.
	WANG X . Optimization and application of privacy budget in differential privacy protection[D]. Nanjing:Nanjing University of Posts and Telecommunications, 2019.