基于目标扰动的AdaBoost算法

doi:10.11959/j.issn.1000-436x.2023028

通信学报 ›› 2023, Vol. 44 ›› Issue (2): 198-209.doi: 10.11959/j.issn.1000-436x.2023028

基于目标扰动的AdaBoost算法

张淑芬¹^,²^,³, 董燕灵¹^,²^,⁴, 徐精诚¹^,²^,⁴, 王豪石¹^,²^,⁴

¹ 华北理工大学理学院，河北唐山 063210
² 河北省数据科学与应用重点实验室，河北唐山 063210
³ 唐山市大数据安全与智能计算重点实验室，河北唐山 063210
⁴ 唐山市数据科学重点实验室，河北唐山 063210

修回日期:2022-12-25 出版日期:2023-02-25 发布日期:2023-02-01
作者简介:张淑芬（1972− ），女，河北唐山人，华北理工大学教授、硕士生导师，主要研究方向为云计算、智能信息处理、数据安全与隐私保护
董燕灵（1998− ），女，浙江宁波人，华北理工大学硕士生，主要研究方向为数据安全与隐私保护
徐精诚（1996− ），男，江苏常州人，华北理工大学硕士生，主要研究方向为数据安全与隐私保护
王豪石（1997− ），男，河北邢台人，华北理工大学硕士生，主要研究方向为数据安全与隐私保护
基金资助:
国家自然科学基金资助项目(U20A20179)

AdaBoost algorithm based on target perturbation

Shufen ZHANG¹^,²^,³, Yanling DONG¹^,²^,⁴, Jingcheng XU¹^,²^,⁴, Haoshi WANG¹^,²^,⁴

¹ College of Science, North China University of Science and Technology, Tangshan 063210, China
² Hebei Key Laboratory of Data Science and Application, Tangshan 063210, China
³ Tangshan Key Laboratory of Big Data Security and Intelligent Computing, Tangshan 063210, China
⁴ Tangshan Key Laboratory of Data Science, Tangshan 063210, China

Revised:2022-12-25 Online:2023-02-25 Published:2023-02-01
Supported by:
The National Natural Science Foundation of China(U20A20179)

摘要/Abstract

摘要：

针对 AdaBoost 算法的多轮迭过程会放大为实现差分隐私保护而添加的噪声，从而导致模型收敛缓慢、数据可用性大幅降低的问题，提出了一种基于目标扰动的AdaBoost算法——DPAda，采用目标扰动的方式对样本权值进行加噪，精确计算其敏感度，并赋予其动态的隐私预算。为了解决噪声叠加过多的问题，提出基于摆动数列、随机响应和改进的随机响应3种噪声注入算法。实验结果表明，与DPAda_Random算法和DPAda_Swing算法相比，DPAda_Improved算法能实现数据的隐私保护，拥有更高的分类准确率，优于其他差分隐私AdaBoost算法，并能解决连续加噪带来的噪声过大的问题。

关键词: 差分隐私, 摆动数列, 随机响应, 隐私预算分配, AdaBoost算法

Abstract:

Aiming at the problem that the multi-round iteration process in the AdaBoost algorithm will amplify the noise added to achieve differential privacy protection, which leads to slow model convergence and greatly reduced data availability, an AdaBoost algorithm based on target perturbation—DPAda was proposed.Target perturbation was used to add noise to sample weights, accurately calculated their sensitivity, and a dynamic privacy budget was given.In order to solve the problem of excessive noise superposition, three noise injection algorithms based on swing sequence, random response and improved random response were proposed.The experimental results show that compared with DPAda_Random and DPAda_Swing, DPAda_Improved achieves the privacy protection of data, has higher classification accuracy, as well as better than other differential privacy AdaBoost algorithm, and can also solve the problem of excessive noise caused by continuous noise addition.

Key words: differential privacy, swing sequence, random response, privacy budget allocation, AdaBoost algorithm

中图分类号:

TP309.2

张淑芬, 董燕灵, 徐精诚, 王豪石. 基于目标扰动的AdaBoost算法[J]. 通信学报, 2023, 44(2): 198-209.

Shufen ZHANG, Yanling DONG, Jingcheng XU, Haoshi WANG. AdaBoost algorithm based on target perturbation[J]. Journal on Communications, 2023, 44(2): 198-209.

图/表 8

图1

表1

主要符号"

符号	含义
D_k	第k轮样本权值分布
w_{k ,i}	第k轮迭代中第i个样本权值
Δf	全局敏感度
G_k(x)	第k轮弱学习器
G (x)	最终分类器
${\tilde{w}}_{k, i}$	加噪后的w_k,i
${\bar{w}}_{k}$	第k轮样本权值均值
e_k	第k轮分类误差率
α_k	第k轮弱学习器权重
ε	隐私预算

表1

表2

图2

图3

图4

图5

图6

参考文献［30］

［1］	陈天莹, 陈剑锋．大数据环境下的智能数据脱敏系统［J］．通信技术, 2016,49(7): 915-922．
	CHEN T Y , CHEN J F ． Intelligent data masking system for big data productive environment［J］． Communications Technology, 2016,49(7): 915-922．
［2］	SWEENEY L ． k-anonymity:a model for protecting privacy［J］． International Journal of Uncertainty,Fuzziness and Knowledge-Based Systems, 2002,10(5): 557-570．
［3］	GENTRY C , ． Fully homomorphic encryption using ideal lattices［C］// Proceedings of the 41st Annual ACM Symposium on Theory of Computing． New York:ACM Press, 2009: 169-178．
［4］	DWORK C , ． Differential privacy［C］// Proceedings of the 33rd International Conference on Automata,Languages and Programming (ICALP’06)． Berlin:Springer, 2006: 1-12．
［5］	DWORK C , AGRAWAL M , DU D ,et al． Differential Privacy:a survey of results［C］// International Conference on Theory and Applications of Models of Computation． Berlin:Springer, 2008: 1-19．
［6］	DWORK C , LEI J ． Differential privacy and robust statistics［C］// Proceedings of the 41st Annual ACM Symposium on Theory of Computing． New York:ACM Press, 2009: 371-380．
［7］	DWORK C , MCSHERRY F , NISSIM K ,et al． Calibrating noise to sensitivity in private data analysis［C］// Theory of Cryptography． Berlin:Springer, 2006: 265-284．
［8］	MCSHERRY F , TALWAR K ． Mechanism design via differential privacy［C］// Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science． Piscataway:IEEE Press, 2007: 94-103．
［9］	DWORK C , ROTH A ． The algorithmic foundations of differential privacy［J］． Foundations and Trends in Theoretical Computer Science, 2013,9(3/4): 211-407．
［10］	JAGANNATHAN G , PILLAIPAKKAMNATT K , WRIGHT R N ． A practical differentially private random decision tree classifier［C］// Proceedings of 2009 IEEE International Conference on Data Mining Workshops． Piscataway:IEEE Press, 2009: 114-121．
［11］	PATIL A , SINGH S ． Differential private random forest［C］// Proceedings of 2014 International Conference on Advances in Computing,Communications and Informatics (ICACCI)． Piscataway:IEEE Press, 2014: 2623-2630．
［12］	FLETCHER S , ISLAM M Z ． A differentially private decision forest［J］． Expert Systems with Applications, 2017,78: 16-31．
［13］	穆海蓉, 丁丽萍, 宋宇宁 ,等． DiffPRFs:一种面向随机森林的差分隐私保护算法［J］．通信学报, 2016,37(9): 175-182．
	MU H R , DING L P , SONG Y N ,et al． DiffPRFs:random forest under differential privacy［J］． Journal on Communications, 2016,37(9): 175-182．
［14］	李远航, 陈先来, 刘莉 ,等．面向差分隐私保护的随机森林算法［J］．计算机工程, 2020,46(1): 93-101．
	LI Y H , CHEN X L , LIU L ,et al． Random forest algorithm for differential privacy protection［J］． Computer Engineering, 2020,46(1): 93-101．
［15］	邓蔚, 陈秀婷, 张清华 ,等．基于树模型的差分隐私保护算法［J］．重庆邮电大学学报(自然科学版), 2020,32(5): 848-856．
	DENG W , CHEN X T , ZHANG Q H ,et al． Differential privacy protection algorithms based on tree model［J］． Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2020,32(5): 848-856．
［16］	FLETCHER S , ISLAM M Z ． Differentially private random decision forests using smooth sensitivity［J］． Expert Systems With Applications, 2017,78: 16-31．
［17］	GUAN Z T , SUN X W , SHI L Y ,et al． A differentially private greedy decision forest classification algorithm with high utility［J］． Computers＆ Security, 2020,96:101930．
［18］	FLETCHER S , ISLAM M Z ． A differentially private random decision forest using reliable signal-to-noise ratios［C］// Australasian Joint Conference on Artificial Intelligence． Berlin:Springer, 2015: 192-203．
［19］	LI X X , LIU J , LIU S F ,et al． Differentially private ensemble learning for classification［J］． Neurocomputing, 2021,430: 34-46．
［20］	GAMBS S , KéGL B , A?MEUR E , ． Privacy-preserving boosting［J］． Data Mining and Knowledge Discovery, 2007,14(1): 131-170．
［21］	LI Y , BAI C X , REDDY C K ． A distributed ensemble approach for mining healthcare data under privacy constraints［J］． Information Sciences, 2016,330: 245-259．
［22］	沈思倩．关于差分隐私保护分类算法的研究［D］．南京:南京航空航天大学, 2017．
	SHEN S Q ． Research on the differential privacy classification algorithms［D］． Nanjing:Nanjing University of Aeronautics and Astronautics, 2017．
［23］	贾俊杰, 邱万勇, 马慧芳．差分隐私保护约束下集成分类算法的研究［J］．信息安全学报, 2021,6(4): 106-118．
	JIA J J , QIU W Y , MA H F ． Research on an ensemble classification algorithm under differential privacy［J］． Journal of Cyber Security, 2021,6(4): 106-118．
［24］	LI Q B , WU Z M , WEN Z Y ,et al． Privacy-preserving gradient boosting decision trees［J］． Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(1): 784-791．
［25］	FREUND Y , SCHAPIRE R E ． A decision-theoretic generalization of on-line learning and an application to boosting［J］． Journal of Computer and System Sciences, 1997,55(1): 119-139．
［26］	FREUND Y , SCHAPIRE R E ． Experiments with a new Boosting algorithm［C］// Proceedings of the 13th Conference on Machine Learning．San Francisco:Morgan Kaufmann Publishers Inc． , 1996: 148-156．
［27］	曹莹, 苗启广, 刘家辰 ,等． AdaBoost 算法研究进展与展望［J］．自动化学报, 2013,39(6): 745-758．
	CAO Y , MIAO Q G , LIU J C ,et al． Advance and prospects of AdaBoost algorithm［J］． Acta Automatica Sinica, 2013,39(6): 745-758．
［28］	FRIEDMAN J , HASTIE T , TIBSHIRANI R ． Additive logistic regression:a statistical view of boosting［J］． The Annals of Statistics, 2000,28(2): 337-407．
［29］	燕会霞．简谐振动与摆动数列的通项公式［J］．数学教学, 2006(9): 19-20．
	YAN H X ． General formula of simple harmonic vibration and swing sequence［J］． Mathematics teaching, 2006(9): 19-20．
［30］	邱万勇．基于差分隐私的集成分类算法及应用研究［D］．兰州:西北师范大学, 2021．
	QIU W Y ． A study of ensemble classification algorithms and applications based on differential privacy［D］． Lanzhou:Northwest Normal University, 2021．

数据集名称	样本数	连续型特征数	离散型特征数	类数
Adult_train	325 61	6	8	2
Adult_test	162 81	6	8	2
Bank Marketing	411 88	10	10	2

[1]	余晟兴, 陈泽凯, 陈钟, 刘西蒙. DAGUARD：联邦学习下的分布式后门攻击防御方案[J]. 通信学报, 2023, 44(5): 110-122.
[2]	冯涛, 陈李秋, 方君丽, 石建明. 基于本地化差分隐私和属性基可搜索加密的区块链数据共享方案[J]. 通信学报, 2023, 44(5): 224-233.
[3]	汤凌韬, 王迪, 刘盛云. 面向非独立同分布数据的联邦学习数据增强方案[J]. 通信学报, 2023, 44(1): 164-176.
[4]	袁程胜, 郭强, 付章杰. 基于差分隐私的深度伪造指纹检测模型版权保护算法[J]. 通信学报, 2022, 43(9): 181-193.
[5]	王瀚仪, 李效光, 毕文卿, 陈亚虹, 李凤华, 牛犇. 多级本地化差分隐私算法推荐框架[J]. 通信学报, 2022, 43(8): 52-64.
[6]	张勇, 李丹丹, 韩璐, 黄小红. 隐私保护的群体感知数据交易算法[J]. 通信学报, 2022, 43(5): 1-13.
[7]	康海燕, 冀源蕊. 基于本地化差分隐私的联邦学习方法研究[J]. 通信学报, 2022, 43(10): 94-105.
[8]	晏燕, 丛一鸣, Adnan Mahmood, 盛权政. 基于深度学习的位置大数据统计发布与隐私保护方法[J]. 通信学报, 2022, 43(1): 203-216.
[9]	陈思, 付安民, 苏铓, 孙怀江. 基于差分隐私的轨迹隐私保护方案[J]. 通信学报, 2021, 42(9): 54-64.
[10]	李洪涛, 任晓宇, 王洁, 马建峰. 基于差分隐私的连续位置隐私保护机制[J]. 通信学报, 2021, 42(8): 164-175.
[11]	蔡剑平, 刘西蒙, 熊金波, 应作斌, 吴英杰. 差分隐私下多重一致性约束问题的逼近方法[J]. 通信学报, 2021, 42(6): 107-117.
[12]	朱素霞, 刘抒伦, 孙广路. 基于相对熵和K-means的形状相似差分隐私轨迹保护机制[J]. 通信学报, 2021, 42(2): 113-123.
[13]	刘杨, 李珺, 陈文韵, 彭木根. 面向6G的雾无线接入网内生安全数据共享机制研究[J]. 通信学报, 2021, 42(1): 67-78.
[14]	叶阿勇,孟玲玉,赵子文,刁一晴,张娇美. 基于预测和滑动窗口的轨迹差分隐私保护机制[J]. 通信学报, 2020, 41(4): 123-133.
[15]	周异辉,鲁来凤,吴振强. 随机响应机制效用优化研究[J]. 通信学报, 2019, 40(6): 74-81.

基于目标扰动的AdaBoost算法

AdaBoost algorithm based on target perturbation

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献［30］

相关文章 15

Metrics

推荐阅读 0

基于目标扰动的AdaBoost算法

AdaBoost algorithm based on target perturbation

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 ［30］

相关文章 15

Metrics

推荐阅读 0

参考文献［30］