神经网络后门攻击研究

doi:10.11959/j.issn.2096-109x.2021053

网络与信息安全学报 ›› 2021, Vol. 7 ›› Issue (3): 46-58.doi: 10.11959/j.issn.2096-109x.2021053

• 专栏Ⅰ：神经网络技术应用 • 上一篇下一篇

神经网络后门攻击研究

谭清尹¹, 曾颖明², 韩叶¹, 刘一静¹, 刘哲理¹

¹ 南开大学网络空间安全学院，天津 300350
² 北京计算机技术及应用研究所，北京 100081

修回日期:2020-07-02 出版日期:2021-06-15 发布日期:2021-06-01
作者简介:谭清尹（1998- ），男，重庆人，南开大学硕士生，主要研究方向为数据隐私保护、移动安全及人工智能安全
曾颖明（1985- ），男，江西丰城人，北京计算机技术及应用研究所高级工程师，主要研究方向为网络安全防护、网络攻防对抗、人工智能算法安全
韩叶（1998- ），女，河南平顶山人，南开大学硕士生，主要研究方向为数据隐私保护、密文数据库
刘一静（1998- ），女，天津人，南开大学硕士生，主要研究方向为数据隐私保护、密文数据库
刘哲理（1978- ），男，山东潍坊人，南开大学副教授、博士生导师，主要研究方向为基于密码学的数据隐私保护、密文数据库、密文集合运算、差分隐私
基金资助:
国家自然科学基金(61672300);天津市人工智能重点专项(18ZXZNGX00140)

Survey on backdoor attacks targeted on neural network

Qingyin TAN¹, Yingming ZENG², Ye HAN¹, Yijing LIU¹, Zheli LIU¹

¹ College of Cyber Science, Nankai University,Tianjin 300350,China
² Beijing Computer Technology and Application Research Institute, Beijing 100081, China

Revised:2020-07-02 Online:2021-06-15 Published:2021-06-01
Supported by:
The National Natural Science Foundation of China(61672300);The Artificial Intelligence Key Special Project of Tianjin(18ZXZNGX00140)

摘要/Abstract

摘要：

针对现有的神经网络后门攻击研究工作，首先介绍了神经网络后门攻击的相关概念；其次，从研究发展历程、典型工作总结、分类情况3个方面对神经网络后门攻击研究现状进行了说明；然后，对典型的后门植入策略进行了详细介绍；最后，对研究现状进行了总结并对未来的研究趋势进行了展望。

关键词: 人工智能安全, 深度学习, 神经网络, 神经网络后门

Abstract:

According to existing neural network backdoor attack research works, the concept of neural network backdoor attack is first introduced.Secondly, the research status of neural network backdoor attack is explained from three aspects: research development, summary of typical work and classification.Then, some typical backdoor attack strategies are analyzed in detail.Finally, the research status is summarized and the future research directions are discussed.

Key words: artificial intelligence security, deep learning, neural networks, neural network backdoor

中图分类号:

TP309.2

谭清尹, 曾颖明, 韩叶, 刘一静, 刘哲理. 神经网络后门攻击研究[J]. 网络与信息安全学报, 2021, 7(3): 46-58.

Qingyin TAN, Yingming ZENG, Ye HAN, Yijing LIU, Zheli LIU. Survey on backdoor attacks targeted on neural network[J]. Chinese Journal of Network and Information Security, 2021, 7(3): 46-58.

图/表 1

参考文献 41

[1]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems. 2012: 1097-1105.
[2]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[3]	GRAVES A , MOHAMED A , HINTON G . Speech recognition with deep recurrent neural networks[C]// 2013 IEEE International Conference on Acoustics,Speech and Signal Processing. 2013: 6645-6649.
[4]	HERMANN K M , BLUNSOM P . Multilingual distributed representations without word alignment[J]. arXiv preprint arXiv:1312.6173, 2013.
[5]	BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014.
[6]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Playing atari with deep reinforcement learning[J]. arXiv preprint arXiv:1312.5602, 2013.
[7]	PARKHI O M , VEDALDI A , ZISSERMAN A ,et al. Deep face recognition[C]// Proceedings of the British Machine Vision Conference (BMVC). 2015.
[8]	SUN Y , WANG X , TANG X . Deep learning face representation from predicting 10 000 classes[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014: 1891-1898.
[9]	CHEN C , SEFF A , KORNHAUSER A ,et al. Deepdriving:learning affordance for direct perception in autonomous driving[C]// Proceedings of the IEEE International Conference on Computer Vision. 2015: 2722-2730.
[10]	SZEGEDY C , ZAREMBA W , SUTSKEVER I ,et al. Intriguing properties of neural networks[J]. arXiv preprint arXiv:1312.6199, 2013.
[11]	GONG Y , LI B , POELLABAUER C ,et al. Real-time adversarial attacks[J]. arXiv preprint arXiv:1905.13399, 2019.
[12]	GU T , DOLAN-GAVITT B , GARG S . BadNets:Identifying vulnerabilities in the machine learning model supply chain[J]. arXiv preprint arXiv:1708.06733, 2017.
[13]	LIU Y , MA S , AAFER Y ,et al. Trojaning attack on neural networks[R]. 2017.
[14]	SHAFAHI A , HUANG W R , NAJIBI M ,et al. Poison Frogs! targeted clean-label poisoning attacks on neural networks[C]// Advances in Neural Information Processing Systems. 2018: 6103-6113.
[15]	ZOU M , SHI Y , WANG C ,et al. PoTrojan:powerful neural-level trojan designs in deep learning models[J]. arXiv preprint arXiv:1802.03043, 2018.
[16]	ZHU C , HUANG W R , SHAFAHI A ,et al. Transferable clean-label attack poisoning attacks on deep neural nets[J]. arXiv Preprint arXiv:1905.05897, 2019.
[17]	BAGDASARYAN E , VEIT A , HUA Y ,et al. How to backdoor federated learning[J]. arXiv preprint arXiv:1807.00459, 2018.
[18]	SUN Z , KAIROUZ P , SURESH A T ,et al. Can you really backdoor federated learning?[J]. arXiv Preprint arXiv:1911.07963, 2019.
[19]	YANG Z , IYER N , REIMANN J ,et al. Design of intentional backdoors in sequential models[J]. arXiv preprint arXiv:1902.09972, 2019.
[20]	KIOURTI P , WARDEGA K , JHA S ,et al. TrojDRL:trojan attacks on deep reinforcement learning agents[J]. arXiv Preprint arXiv:1903.06638, 2019.
[21]	YAO Y , LI H , ZHENG H ,et al. Regula sub-rosa:latent backdoor attacks on deep neural networks[J]. arXiv preprint arXiv:1905.10447, 2019.
[22]	YAO Y , LI H , ZHENG H ,et al. Latent backdoor attacks on deep neural networks[C]// Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019: 2041-2055.
[23]	XU G , LI H , LIU S ,et al. VerifyNet:secure and verifiable federated learning[J]. IEEE Transactions on information forensics and Security, 2019,15: 911-926.
[24]	TAN T J L , SHOKRI R . Bypassing backdoor detection algorithms in deep learning[J]. arXiv Preprint arXiv:1905.13409, 2019.
[25]	JI Y , ZHANG X , WANG T . Backdoor attacks against learning systems[C]// 2017 IEEE Conference on Communications and Network Security (CNS). 2017: 1-9.
[26]	BIGGIO B , NELSON B , LASKOV P . Poisoning attacks against support vector machines[J]. arXiv Preprint arXiv:1206.6389, 2012.
[27]	HUANG L , JOSEPH A D , NELSON B ,et al. Adversarial machine learning[C]// Proceedings of the 4th ACM Workshop on Security and Artificial intelligence. 2011: 43-58.
[28]	MAHLOUJIFAR S , MAHMOODY M , MOHAMMED A . Multi-party poisoning through generalized $ p $-tampering[J]. arXiv preprint arXiv:1809.03474, 2018.
[29]	CHEN X , LIU C , LI B ,et al. Targeted backdoor attacks on deep learning systems using data poisoning[J]. arXiv preprint arXiv:1712.05526, 2017.
[30]	DAI J , CHEN C , LI Y . A backdoor attack against LSTM-based text classification systems[J]. IEEE Access, 2019,7: 138872-138878.
[31]	DUMFORD J , SCHEIRER W . Backdooring convolutional neural networks via targeted weight perturbations[J]. arXiv Preprint arXiv:1812.03128, 2018.
[32]	JI Y , ZHANG X , JI S ,et al. Model-reuse attacks on deep learning systems[C]// Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2018: 349-363.
[33]	PAN S J , YANG Q . A survey on transfer learning[J]. IEEE Transactions on knowledge and data engineering, 2009,22(10): 1345-1359.
[34]	刘全, 翟建伟, 章宗长 ,等. 深度强化学习综述[J]. 计算机学报, 2018,41(01): 1-27.
	LIU Q , ZHAI J W , ZHANG Z C ,et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018,4(1): 1-27.
[35]	Decentralized ML[EB].
[36]	KONE?NY J , MCMAHAN H B , YU F X ,et al. Federated learning:Strategies for improving communication efficiency[J]. arXiv Preprint arXiv:1610.05492, 2016.
[37]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[J]. arXiv Preprint arXiv:1602.05629, 2016.
[38]	TRAN B , LI J , MADRY A . Spectral signatures in backdoor attacks[C]// Advances in Neural Information Processing Systems. 2018: 8000-8010.
[39]	CHEN B , CARVALHO W , BARACALDO N ,et al. Detecting backdoor attacks on deep neural networks by activation clustering[J]. arXiv Preprint arXiv:1811.03728, 2018.
[40]	LIU K , DOLAN-GAVITT B , GARG S . Fine-pruning:Defending against backdooring attacks on deep neural networks[C]// International Symposium on Research in Attacks,Intrusions,and Defenses. Springer,Cham, 2018: 273-294.
[41]	PANG R . The tale of evil twins:adversarial inputs versus backdoored models[J]. arXiv preprint arXiv:1911.01559, 2019.

方法	时间/年份	攻击实现	目标场景	现实性	隐蔽性
BadNets^[12]	2017	数据中毒	外包训练	低	低
Trojaning Attack^[13]	2017	模型操作	迁移学习	中	中
PLMs Backdoor Attack^[25]	2017	模型中毒	迁移学习	中	高
Backdoor Attack^[29]	2017	数据中毒	—	低	中
Clean-Label Attack^[14]	2018	数据中毒	迁移学习	低	高
PoTrojan^[15]	2018	模型操作	迁移学习	中	中
Federated learning backdoor^[17]	2018	模型中毒	联邦学习	中	高
Model-Reuse^[32]	2018	模型中毒	迁移学习	高	高
WeightPerturbations^[31]	2018	模型操作	—	中	高
Backdoors in sequential models^[19]	2019	数据中毒	深度强化学习	中	中
TrojDRL^[20]	2019	数据中毒	深度强化学习	中	高
Transferable Clean-Label Attack^[16]	2019	数据中毒	迁移学习	中	高
Latent Backdoor^[21-22]	2019	模型中毒	迁移学习	高	优
Really Backdoor Federated Learning^[18]	2019	模型中毒	联邦学习	高	高
Bypassing Detection Backdoor^[24]	2019	数据中毒	—	中	高
Text classification systems backdoor^[30]	2019	数据中毒	—	中	高
注：“—”指该攻击针对的场景为作者设定的特定场景。

神经网络后门攻击研究

Survey on backdoor attacks targeted on neural network

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 1

参考文献 41

相关文章 15

Metrics

推荐阅读 0

[1]	陈立全, 朱宇航, 王宇, 秦中元, 马旸. 基于混沌神经网络和C-MD结构的带密钥哈希函数[J]. 网络与信息安全学报, 2023, 9(3): 1-15.
[2]	冯冠云, 付才, 吕建强, 韩兰胜. 基于操作注意力和数据增强的内部威胁检测[J]. 网络与信息安全学报, 2023, 9(3): 102-112.
[3]	李晓萌, 郭玳豆, 卓训方, 姚恒, 秦川. 载体独立的抗屏摄信息膜叠加水印算法[J]. 网络与信息安全学报, 2023, 9(3): 135-149.
[4]	蔡召, 荆涛, 任爽. 以太坊钓鱼诈骗检测技术综述[J]. 网络与信息安全学报, 2023, 9(2): 21-32.
[5]	侯泽洲, 任炯炯, 陈少真. 基于神经网络区分器的SIMON-like算法参数安全性评估[J]. 网络与信息安全学报, 2023, 9(2): 154-163.
[6]	沈晓晨, 葛寅辉, 陈波, 于泠. 人工智能安全知识图谱构建技术研究[J]. 网络与信息安全学报, 2023, 9(2): 164-174.
[7]	谢绒娜, 马铸鸿, 李宗俞, 田野. 基于卷积神经网络的加密流量分类方法[J]. 网络与信息安全学报, 2022, 8(6): 84-91.
[8]	章登勇, 文凰, 李峰, 曹鹏, 向凌云, 杨高波, 丁湘陵. 基于双分支网络的图像修复取证方法[J]. 网络与信息安全学报, 2022, 8(6): 110-122.
[9]	林佳滢, 周文柏, 张卫明, 俞能海. 空域频域相结合的唇型篡改检测方法[J]. 网络与信息安全学报, 2022, 8(6): 146-155.
[10]	单棣斌, 杜学绘, 王文娟, 刘敖迪, 王娜. 基于GNN双源学习的访问控制关系预测方法[J]. 网络与信息安全学报, 2022, 8(5): 40-55.
[11]	陈浩, 王锋, 张卫明, 俞能海. 载体独立的深度光照水印算法[J]. 网络与信息安全学报, 2022, 8(4): 110-118.
[12]	林点, 潘理, 易平. 面向图像识别的卷积神经网络鲁棒性研究进展[J]. 网络与信息安全学报, 2022, 8(3): 111-122.
[13]	王馨雅, 华光, 江昊, 张海剑. 深度学习模型的版权保护研究综述[J]. 网络与信息安全学报, 2022, 8(2): 1-14.
[14]	陈晋音, 吴长安, 郑海斌. 基于softmax激活变换的对抗防御方法[J]. 网络与信息安全学报, 2022, 8(2): 48-63.
[15]	邱宝琳, 易平. 基于多维特征图知识蒸馏的对抗样本防御方法[J]. 网络与信息安全学报, 2022, 8(2): 88-99.