深度学习中对抗样本的构造及防御研究

doi:10.11959/j.issn.2096-109x.2020016

网络与信息安全学报 ›› 2020, Vol. 6 ›› Issue (2): 1-11.doi: 10.11959/j.issn.2096-109x.2020016

• 综述 • 下一篇

深度学习中对抗样本的构造及防御研究

段广晗¹,马春光²(),宋蕾¹,武朋²

¹ 哈尔滨工程大学计算机科学与技术学院，黑龙江哈尔滨 150001
² 山东科技大学计算机科学与工程学院，山东青岛 266590

修回日期:2019-08-20 出版日期:2020-04-15 发布日期:2020-04-23
作者简介:段广晗（1994– ），男，黑龙江海伦人，哈尔滨工程大学博士生，主要研究方向为深度学习、对抗样本、机器学习|马春光（1974– ），男，黑龙江双城人，山东科技大学教授、博士生导师，主要研究方向为密码学、数据安全与隐私、人工智能安全与隐私、区块链技术与应用|宋蕾（1989– ），女，黑龙江牡丹江人，哈尔滨工程大学博士生，主要研究方向为机器学习安全与隐私保护、云计算、网络安全|武朋（1974– ），女，黑龙江齐齐哈尔人，山东科技大学讲师，主要研究方向为网络安全、隐私保护
基金资助:
国家自然科学基金资助项目(61472097);国家自然科学基金资助项目(61932005);国家自然科学基金资助项目(U1936112);黑龙江省自然科学基金资助项目(JJ2019LH1770)

Research on structure and defense of adversarial example in deep learning

Guanghan DUAN¹,Chunguang MA²(),Lei SONG¹,Peng WU²

¹ College of Computer Science and Technology,Harbin Engineering University,Harbin 150001,China
² College of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao 266590,China

Revised:2019-08-20 Online:2020-04-15 Published:2020-04-23
Supported by:
The National Natural Science Foundation of China(61472097);The National Natural Science Foundation of China(61932005);The National Natural Science Foundation of China(U1936112);The Natural Science Foundation of Heilongjiang Province(JJ2019LH1770)

摘要/Abstract

摘要：

随着深度学习技术在计算机视觉、网络安全、自然语言处理等领域的进一步发展，深度学习技术逐渐暴露了一定的安全隐患。现有的深度学习算法无法有效描述数据本质特征，导致算法面对恶意输入时可能无法给出正确结果。以当前深度学习面临的安全威胁为出发点，介绍了深度学习中的对抗样本问题，梳理了现有的对抗样本存在性解释，回顾了经典的对抗样本构造方法并对其进行了分类，简述了近年来部分对抗样本在不同场景中的应用实例，对比了若干对抗样本防御技术，最后归纳对抗样本研究领域存在的问题并对这一领域的发展趋势进行了展望。

关键词: 对抗样本, 深度学习, 安全威胁, 防御技术

Abstract:

With the further promotion of deep learning technology in the fields of computer vision,network security and natural language processing,which has gradually exposed certain security risks.Existing deep learning algorithms can not effectively describe the essential characteristics of data or its inherent causal relationship.When the algorithm faces malicious input,it often fails to give correct judgment results.Based on the current security threats of deep learning,the adversarial example problem and its characteristics in deep learning applications were introduced,hypotheses on the existence of adversarial examples were summarized,classic adversarial example construction methods were reviewed and recent research status in different scenarios were summarized,several defense techniques in different processes were compared,and finally the development trend of adversarial example research were forecasted.

Key words: adversarial example, deep learning, security threat, defense technology

中图分类号:

TP309

段广晗,马春光,宋蕾,武朋. 深度学习中对抗样本的构造及防御研究[J]. 网络与信息安全学报, 2020, 6(2): 1-11.

Guanghan DUAN,Chunguang MA,Lei SONG,Peng WU. Research on structure and defense of adversarial example in deep learning[J]. Chinese Journal of Network and Information Security, 2020, 6(2): 1-11.

图/表 7

图1

图2

图3

图4

表1

图5

图6

参考文献 52

[1]	SZEGEDY C , VANHOUCKE V , IOFFE S ,et al. Rethinking the inception architecture for computer vision[C]// The IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2818-2826.
[2]	TANG T A , MHAMDI L , MCLERNON D ,et al. Deep learning approach for network intrusion detection in software defined networking[C]// 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM). 2016: 258-263.
[3]	COLLOBERT R , WESTON J . A unified architecture for natural language processing:deep neural networks with multitask learning[C]// The 25th International Conference on Machine Learning. 2008: 160-167.
[4]	CHEN C , SEFF A , KORNHAUSER A ,et al. Deepdriving:learning affordance for direct perception in autonomous driving[C]// The IEEE International Conference on Computer Vision. 2015: 2722-2730.
[5]	CHING T , HIMMELSTEIN D S , BEAULIEU-JONES B K ,et al. Opportunities and obstacles for deep learning in biology and medicine[J]. Journal of The Royal Society Interface, 2018,15(141).
[6]	SZEGEDY C , ZAREMBA W , SUTSKEVER I ,et al. Intriguing properties of neural networks[J]. arXiv preprint arXiv:1312.6199, 2013
[7]	KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial examples in the physical world[J]. arXiv preprint arXiv:1607.02533, 2016
[8]	ALZANTOT M , SHARMA Y , ELGOHARY A ,et al. Generating natural language adversarial examples[J]. arXiv preprint arXiv:1804.07998, 2018
[9]	QIN Y , CARLINI N , GOODFELLOW I ,et al. Imperceptible,robust,and targeted adversarial examples for automatic speech recognition[J]. arXiv preprint arXiv:1903.10346, 2019
[10]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553):436.
[11]	PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples[J]. arXiv preprint arXiv:1605.07277, 2016
[12]	PAPERNOT N , MCDANIEL P , JHA S ,et al. The limitations of deep learning in adversarial settings[C]// The 1st IEEE European Symposium on Security and Privacy. 2016.
[13]	宋蕾, 马春光, 段广晗 . 机器学习安全及隐私保护研究进展[J]. 网络与信息安全学报, 2018,4(8): 1-11.
	SONG L , MA C G , DUAN G H . Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018,4(8): 1-11.
[14]	GU S , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples[J]. arXiv preprint arXiv:1412.5068, 2014
[15]	GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples[C]// 2015 International Conference on Learning Representations. 2015: 1-10.
[16]	TABACOF P , VALLE E . Exploring the space of adversarial images[J]. arXiv preprint arXiv:1510.05328, 2015
[17]	TRAM`ER F , PAPERNOT N , GOODFELLOW I ,et al. The space of transferable adversarial examples[J]. arXiv preprint arXiv:1704.03453, 2017
[18]	KROTOV D , HOPFIELD J J . Dense associative memory is robust to adversarial inputs[J]. arXiv preprint arXiv:1701.00939, 2017
[19]	LUO Y , BOIX X , ROIG G ,et al. Foveation-based mechanisms alleviate adversarial examples[J]. arXiv preprint arXiv:1511.06292, 2015
[20]	TANAY T , GRIFFIN L . A boundary tilting perspective on the phenomenon of adversarial examples[J]. arXiv preprint arXiv:1608.07690, 2016
[21]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Universal adversarial perturbations[C]// The IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1765-1773.
[22]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Analysis of universal adversarial perturbations[J]. arXiv preprint arXiv:1705.09554, 2017
[23]	TRAM`ER F , KURAKIN A , PAPERNOT N ,et al. Ensemble adversarial training:attacks and defenses[J]. arXiv preprint arXiv:1705.07204, 2017
[24]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Robustness of classifiers to universal perturbations:a geometric perspective[C]// International Conference on Learning Representations. 2018.
[25]	SONG Y , KIM T , NOWOZIN S ,et al. Pixeldefend:leveraging generative models to understand and defend against adversarial examples[J]. arXiv preprint arXiv:1710.10766, 2017
[26]	MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples[C]// The 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 135-147.
[27]	GHOSH P , LOSALKA A , BLACK M J . Resisting adversarial attacks using gaussian mixture variational autoencoders[J]. arXiv preprint arXiv:1806.00081, 2018
[28]	LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with gan[J]. arXiv preprint arXiv:1705.03387, 2017
[29]	GILMER J , METZ L , FAGHRI F ,et al. Adversarial spheres[J]. arXiv preprint arXiv:1801.02774, 2018
[30]	GILMER J , METZ L , FAGHRI F ,et al. The relationship between high-dimensional geometry and adversarial examples[J]. arXiv:1801.02774v3, 2018
[31]	EYKHOLT K , EVTIMOV I , FERNANDES E ,et al. Robust physical-world attacks on deep learning visual classification[C]// The IEEE Conference on Computer Vision and Pattern Recognition. 2018: 1625-1634.
[32]	MOOSAVI-DEZFOOLI S M , FAWZI A , FROSSARD P . Deepfool:a simple and accurate method to fool deep neural networks[C]// The 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016.
[33]	PAPERNOT N , MCDANIEL P , SWAMI A ,et al. Crafting adversarial input sequences for recurrent neural networks[C]// MILCOM 2016-2016 IEEE Military Communications Conference. 2016: 49-54.
[34]	GROSSE K , PAPERNOT N , MANOHARAN P ,et al. Adversarial examples for malware detection[C]// European Symposium on Research in Computer Security. 2017: 62-79.
[35]	RUSSAKOVSKY O , DENG J , SU H ,et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015,115(3): 211-252.
[36]	PAPERNOT N , MCDANIEL P , JHA S ,et al. The limitations of deep learning in adversarial settings[C]// 2016 IEEE European Symposium on Security and Privacy. 2016: 372-387.
[37]	PAPERNOT N , MCDANIEL P , GOODFELLOW I ,et al. Practical black-box attacks against machine learning[C]// The 2017 ACM on Asia Conference on Computer and Communications Security. 2017: 506-519.
[38]	ILYAS A , ENGSTROM L , ATHALYE A ,et al. Black-box adversarial attacks with limited queries and information[J]. arXiv preprint arXiv:1804.08598, 2018
[39]	BALUJA S , FISCHER I . Adversarial transformation networks:Learning to generate adversarial examples[J]. arXiv preprint arXiv:1703.09387, 2017
[40]	XIAO C , LI B , ZHU J Y ,et al. Generating adversarial examples with adversarial networks[C]// The 27th International Joint on Artificial Intelligence Main track. 2019: 3805-3911.
[41]	ZHAO P , FU Z , HU Q ,et al. Detecting adversarial examples via key-based network[J]. arXiv preprint arXiv:1806.00580, 2018
[42]	MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples[C]// The 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 135-147.
[43]	XU W , EVANS D , QI Y . Feature squeezing:detecting adversarial examples in deep neural networks[J]. arXiv preprint arXiv:1704.01155, 2017
[44]	HOSSEIN H , CHEN Y , KANNAN S ,et al. Blocking transferability of adversarial examples in black-box learning systems[J]. arXiv:1703.04318, 2017
[45]	SABOUR S , NICHOLAS F , HINTON G E . Dynamic routing between capsules[C]// Neural Information Processing Systems. 2017.
[46]	NICHOLAS F , SABOUR S , HINTON G . DARCCC:detecting adversaries by reconstruction from class conditional capsules[J]. arXiv preprint arXiv:1811.06969, 2018
[47]	TRAMèR F , KURAKIN A , PAPERNOT N ,et al. Ensemble adversarial raining:attacks and defenses[J]. arXiv:1705.07204, 2017
[48]	SINHA A , CHEN Z , BADRINARAYANAN V ,et al. Gradient adversarial training of neural networks[J]. arXiv preprint arXiv:1806.08028, 2018
[49]	KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale[J]. arXiv preprint arXiv:1611.01236, 2016
[50]	PAPERNOT N , MCDANIEL P , WU X ,et al. Distillation as a defense to adversarial perturbations against deep neural networks[C]// 2016 IEEE Symposium on Security and Privacy. 2016: 582-597.
[51]	HINTON G E , VINYALS O , DEAN J . Distilling the knowledge in a neural network[J]. arXiv:1503.02531,
[52]	LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with GAN[J]. arXiv preprint arXiv:1705.03387, 2017

攻击名称	生成特征	攻击目标	迭代次数	先验知识	适用范围
L-BFGS	优化搜索	有目标	多次	白盒	特异攻击
Deep Fool	优化搜索	无目标	多次	白盒	特异攻击
UAP	优化搜索	无目标	多次	白盒	通用攻击
FGSM	特征构造	无目标	单次	白盒	特异攻击
BIM	特征构造	无目标	多次	白盒	特异攻击
LLC	特征构造	无目标	多次	白盒	特异攻击
JSMA	特征构造	有目标	多次	白盒	特异攻击
PBA	特征构造	有目标＆无目标	多次	黑盒	特异攻击
ATN	生成模型	有目标＆无目标	多次	白盒＆黑盒	特异攻击
AdvGAN	生成模型	有目标	多次	白盒	特异攻击

深度学习中对抗样本的构造及防御研究

Research on structure and defense of adversarial example in deep learning

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 52

相关文章 15

Metrics

推荐阅读 0

[1]	李晓萌, 郭玳豆, 卓训方, 姚恒, 秦川. 载体独立的抗屏摄信息膜叠加水印算法[J]. 网络与信息安全学报, 2023, 9(3): 135-149.
[2]	谢绒娜, 马铸鸿, 李宗俞, 田野. 基于卷积神经网络的加密流量分类方法[J]. 网络与信息安全学报, 2022, 8(6): 84-91.
[3]	章登勇, 文凰, 李峰, 曹鹏, 向凌云, 杨高波, 丁湘陵. 基于双分支网络的图像修复取证方法[J]. 网络与信息安全学报, 2022, 8(6): 110-122.
[4]	林佳滢, 周文柏, 张卫明, 俞能海. 空域频域相结合的唇型篡改检测方法[J]. 网络与信息安全学报, 2022, 8(6): 146-155.
[5]	林点, 潘理, 易平. 面向图像识别的卷积神经网络鲁棒性研究进展[J]. 网络与信息安全学报, 2022, 8(3): 111-122.
[6]	陈晋音, 吴长安, 郑海斌. 基于softmax激活变换的对抗防御方法[J]. 网络与信息安全学报, 2022, 8(2): 48-63.
[7]	邱宝琳, 易平. 基于多维特征图知识蒸馏的对抗样本防御方法[J]. 网络与信息安全学报, 2022, 8(2): 88-99.
[8]	李丽娟, 李曼, 毕红军, 周华春. 基于混合深度学习的多类型低速率DDoS攻击检测方法[J]. 网络与信息安全学报, 2022, 8(1): 73-85.
[9]	秦中元, 贺兆祥, 李涛, 陈立全. 基于图像重构的MNIST对抗样本防御算法[J]. 网络与信息安全学报, 2022, 8(1): 86-94.
[10]	邹德清, 李响, 黄敏桓, 宋翔, 李浩, 李伟明. 基于图结构源代码切片的智能化漏洞检测系统[J]. 网络与信息安全学报, 2021, 7(5): 113-122.
[11]	王正龙, 张保稳. 生成对抗网络研究综述[J]. 网络与信息安全学报, 2021, 7(4): 68-85.
[12]	李炳龙, 佟金龙, 张宇, 孙怡峰, 王清贤, 常朝稳. 基于TensorFlow的恶意代码片段自动取证检测算法[J]. 网络与信息安全学报, 2021, 7(4): 154-163.
[13]	谭清尹, 曾颖明, 韩叶, 刘一静, 刘哲理. 神经网络后门攻击研究[J]. 网络与信息安全学报, 2021, 7(3): 46-58.
[14]	李艳, 刘威, 孙远路. 基于对抗学习的强PUF安全结构研究[J]. 网络与信息安全学报, 2021, 7(3): 115-122.
[15]	超凡, 杨智, 杜学绘, 韩冰. 基于多因素聚类选择的Android应用程序分类风险评估方法[J]. 网络与信息安全学报, 2021, 7(2): 161-173.