深度学习中对抗样本的构造及防御研究

doi:10.11959/j.issn.2096-109x.2020016

网络与信息安全学报 ›› 2020, Vol. 6 ›› Issue (2): 1-11.doi: 10.11959/j.issn.2096-109x.2020016

• 综述 • 下一篇

深度学习中对抗样本的构造及防御研究

段广晗¹,马春光²(),宋蕾¹,武朋²

¹ 哈尔滨工程大学计算机科学与技术学院，黑龙江哈尔滨 150001
² 山东科技大学计算机科学与工程学院，山东青岛 266590

修回日期:2019-08-20 出版日期:2020-04-15 发布日期:2020-04-23
作者简介:段广晗（1994– ），男，黑龙江海伦人，哈尔滨工程大学博士生，主要研究方向为深度学习、对抗样本、机器学习|马春光（1974– ），男，黑龙江双城人，山东科技大学教授、博士生导师，主要研究方向为密码学、数据安全与隐私、人工智能安全与隐私、区块链技术与应用|宋蕾（1989– ），女，黑龙江牡丹江人，哈尔滨工程大学博士生，主要研究方向为机器学习安全与隐私保护、云计算、网络安全|武朋（1974– ），女，黑龙江齐齐哈尔人，山东科技大学讲师，主要研究方向为网络安全、隐私保护
基金资助:
国家自然科学基金资助项目(61472097);国家自然科学基金资助项目(61932005);国家自然科学基金资助项目(U1936112);黑龙江省自然科学基金资助项目(JJ2019LH1770)

Research on structure and defense of adversarial example in deep learning

Guanghan DUAN¹,Chunguang MA²(),Lei SONG¹,Peng WU²

¹ College of Computer Science and Technology,Harbin Engineering University,Harbin 150001,China
² College of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao 266590,China

Revised:2019-08-20 Online:2020-04-15 Published:2020-04-23
Supported by:
The National Natural Science Foundation of China(61472097);The National Natural Science Foundation of China(61932005);The National Natural Science Foundation of China(U1936112);The Natural Science Foundation of Heilongjiang Province(JJ2019LH1770)

摘要/Abstract

摘要：

随着深度学习技术在计算机视觉、网络安全、自然语言处理等领域的进一步发展，深度学习技术逐渐暴露了一定的安全隐患。现有的深度学习算法无法有效描述数据本质特征，导致算法面对恶意输入时可能无法给出正确结果。以当前深度学习面临的安全威胁为出发点，介绍了深度学习中的对抗样本问题，梳理了现有的对抗样本存在性解释，回顾了经典的对抗样本构造方法并对其进行了分类，简述了近年来部分对抗样本在不同场景中的应用实例，对比了若干对抗样本防御技术，最后归纳对抗样本研究领域存在的问题并对这一领域的发展趋势进行了展望。

关键词: 对抗样本, 深度学习, 安全威胁, 防御技术

Abstract:

With the further promotion of deep learning technology in the fields of computer vision,network security and natural language processing,which has gradually exposed certain security risks.Existing deep learning algorithms can not effectively describe the essential characteristics of data or its inherent causal relationship.When the algorithm faces malicious input,it often fails to give correct judgment results.Based on the current security threats of deep learning,the adversarial example problem and its characteristics in deep learning applications were introduced,hypotheses on the existence of adversarial examples were summarized,classic adversarial example construction methods were reviewed and recent research status in different scenarios were summarized,several defense techniques in different processes were compared,and finally the development trend of adversarial example research were forecasted.

Key words: adversarial example, deep learning, security threat, defense technology

中图分类号:

TP309

段广晗,马春光,宋蕾,武朋. 深度学习中对抗样本的构造及防御研究[J]. 网络与信息安全学报, 2020, 6(2): 1-11.

Guanghan DUAN,Chunguang MA,Lei SONG,Peng WU. Research on structure and defense of adversarial example in deep learning[J]. Chinese Journal of Network and Information Security, 2020, 6(2): 1-11.

图/表 7

图1

图2

图3

图4

表1

图5

图6

参考文献 52

[1]	SZEGEDY C , VANHOUCKE V , IOFFE S ,et al. Rethinking the inception architecture for computer vision[C]// The IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2818-2826.
[2]	TANG T A , MHAMDI L , MCLERNON D ,et al. Deep learning approach for network intrusion detection in software defined networking[C]// 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM). 2016: 258-263.
[3]	COLLOBERT R , WESTON J . A unified architecture for natural language processing:deep neural networks with multitask learning[C]// The 25th International Conference on Machine Learning. 2008: 160-167.
[4]	CHEN C , SEFF A , KORNHAUSER A ,et al. Deepdriving:learning affordance for direct perception in autonomous driving[C]// The IEEE International Conference on Computer Vision. 2015: 2722-2730.
[5]	CHING T , HIMMELSTEIN D S , BEAULIEU-JONES B K ,et al. Opportunities and obstacles for deep learning in biology and medicine[J]. Journal of The Royal Society Interface, 2018,15(141).
[6]	SZEGEDY C , ZAREMBA W , SUTSKEVER I ,et al. Intriguing properties of neural networks[J]. arXiv preprint arXiv:1312.6199, 2013
[7]	KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial examples in the physical world[J]. arXiv preprint arXiv:1607.02533, 2016
[8]	ALZANTOT M , SHARMA Y , ELGOHARY A ,et al. Generating natural language adversarial examples[J]. arXiv preprint arXiv:1804.07998, 2018
[9]	QIN Y , CARLINI N , GOODFELLOW I ,et al. Imperceptible,robust,and targeted adversarial examples for automatic speech recognition[J]. arXiv preprint arXiv:1903.10346, 2019
[10]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553):436.
[11]	PAPERNOT N , MCDANIEL P , GOODFELLOW I . Transferability in machine learning:from phenomena to black-box attacks using adversarial samples[J]. arXiv preprint arXiv:1605.07277, 2016
[12]	PAPERNOT N , MCDANIEL P , JHA S ,et al. The limitations of deep learning in adversarial settings[C]// The 1st IEEE European Symposium on Security and Privacy. 2016.
[13]	宋蕾, 马春光, 段广晗 . 机器学习安全及隐私保护研究进展[J]. 网络与信息安全学报, 2018,4(8): 1-11.
	SONG L , MA C G , DUAN G H . Machine learning security and privacy:a survey[J]. Chinese Journal of Network and Information Security, 2018,4(8): 1-11.
[14]	GU S , RIGAZIO L . Towards deep neural network architectures robust to adversarial examples[J]. arXiv preprint arXiv:1412.5068, 2014
[15]	GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples[C]// 2015 International Conference on Learning Representations. 2015: 1-10.
[16]	TABACOF P , VALLE E . Exploring the space of adversarial images[J]. arXiv preprint arXiv:1510.05328, 2015
[17]	TRAM`ER F , PAPERNOT N , GOODFELLOW I ,et al. The space of transferable adversarial examples[J]. arXiv preprint arXiv:1704.03453, 2017
[18]	KROTOV D , HOPFIELD J J . Dense associative memory is robust to adversarial inputs[J]. arXiv preprint arXiv:1701.00939, 2017
[19]	LUO Y , BOIX X , ROIG G ,et al. Foveation-based mechanisms alleviate adversarial examples[J]. arXiv preprint arXiv:1511.06292, 2015
[20]	TANAY T , GRIFFIN L . A boundary tilting perspective on the phenomenon of adversarial examples[J]. arXiv preprint arXiv:1608.07690, 2016
[21]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Universal adversarial perturbations[C]// The IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1765-1773.
[22]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Analysis of universal adversarial perturbations[J]. arXiv preprint arXiv:1705.09554, 2017
[23]	TRAM`ER F , KURAKIN A , PAPERNOT N ,et al. Ensemble adversarial training:attacks and defenses[J]. arXiv preprint arXiv:1705.07204, 2017
[24]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Robustness of classifiers to universal perturbations:a geometric perspective[C]// International Conference on Learning Representations. 2018.
[25]	SONG Y , KIM T , NOWOZIN S ,et al. Pixeldefend:leveraging generative models to understand and defend against adversarial examples[J]. arXiv preprint arXiv:1710.10766, 2017
[26]	MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples[C]// The 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 135-147.
[27]	GHOSH P , LOSALKA A , BLACK M J . Resisting adversarial attacks using gaussian mixture variational autoencoders[J]. arXiv preprint arXiv:1806.00081, 2018
[28]	LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with gan[J]. arXiv preprint arXiv:1705.03387, 2017
[29]	GILMER J , METZ L , FAGHRI F ,et al. Adversarial spheres[J]. arXiv preprint arXiv:1801.02774, 2018
[30]	GILMER J , METZ L , FAGHRI F ,et al. The relationship between high-dimensional geometry and adversarial examples[J]. arXiv:1801.02774v3, 2018
[31]	EYKHOLT K , EVTIMOV I , FERNANDES E ,et al. Robust physical-world attacks on deep learning visual classification[C]// The IEEE Conference on Computer Vision and Pattern Recognition. 2018: 1625-1634.
[32]	MOOSAVI-DEZFOOLI S M , FAWZI A , FROSSARD P . Deepfool:a simple and accurate method to fool deep neural networks[C]// The 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016.
[33]	PAPERNOT N , MCDANIEL P , SWAMI A ,et al. Crafting adversarial input sequences for recurrent neural networks[C]// MILCOM 2016-2016 IEEE Military Communications Conference. 2016: 49-54.
[34]	GROSSE K , PAPERNOT N , MANOHARAN P ,et al. Adversarial examples for malware detection[C]// European Symposium on Research in Computer Security. 2017: 62-79.
[35]	RUSSAKOVSKY O , DENG J , SU H ,et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015,115(3): 211-252.
[36]	PAPERNOT N , MCDANIEL P , JHA S ,et al. The limitations of deep learning in adversarial settings[C]// 2016 IEEE European Symposium on Security and Privacy. 2016: 372-387.
[37]	PAPERNOT N , MCDANIEL P , GOODFELLOW I ,et al. Practical black-box attacks against machine learning[C]// The 2017 ACM on Asia Conference on Computer and Communications Security. 2017: 506-519.
[38]	ILYAS A , ENGSTROM L , ATHALYE A ,et al. Black-box adversarial attacks with limited queries and information[J]. arXiv preprint arXiv:1804.08598, 2018
[39]	BALUJA S , FISCHER I . Adversarial transformation networks:Learning to generate adversarial examples[J]. arXiv preprint arXiv:1703.09387, 2017
[40]	XIAO C , LI B , ZHU J Y ,et al. Generating adversarial examples with adversarial networks[C]// The 27th International Joint on Artificial Intelligence Main track. 2019: 3805-3911.
[41]	ZHAO P , FU Z , HU Q ,et al. Detecting adversarial examples via key-based network[J]. arXiv preprint arXiv:1806.00580, 2018
[42]	MENG D , CHEN H . Magnet:a two-pronged defense against adversarial examples[C]// The 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017: 135-147.
[43]	XU W , EVANS D , QI Y . Feature squeezing:detecting adversarial examples in deep neural networks[J]. arXiv preprint arXiv:1704.01155, 2017
[44]	HOSSEIN H , CHEN Y , KANNAN S ,et al. Blocking transferability of adversarial examples in black-box learning systems[J]. arXiv:1703.04318, 2017
[45]	SABOUR S , NICHOLAS F , HINTON G E . Dynamic routing between capsules[C]// Neural Information Processing Systems. 2017.
[46]	NICHOLAS F , SABOUR S , HINTON G . DARCCC:detecting adversaries by reconstruction from class conditional capsules[J]. arXiv preprint arXiv:1811.06969, 2018
[47]	TRAMèR F , KURAKIN A , PAPERNOT N ,et al. Ensemble adversarial raining:attacks and defenses[J]. arXiv:1705.07204, 2017
[48]	SINHA A , CHEN Z , BADRINARAYANAN V ,et al. Gradient adversarial training of neural networks[J]. arXiv preprint arXiv:1806.08028, 2018
[49]	KURAKIN A , GOODFELLOW I , BENGIO S . Adversarial machine learning at scale[J]. arXiv preprint arXiv:1611.01236, 2016
[50]	PAPERNOT N , MCDANIEL P , WU X ,et al. Distillation as a defense to adversarial perturbations against deep neural networks[C]// 2016 IEEE Symposium on Security and Privacy. 2016: 582-597.
[51]	HINTON G E , VINYALS O , DEAN J . Distilling the knowledge in a neural network[J]. arXiv:1503.02531,
[52]	LEE H , HAN S , LEE J . Generative adversarial trainer:defense to adversarial perturbations with GAN[J]. arXiv preprint arXiv:1705.03387, 2017

攻击名称	生成特征	攻击目标	迭代次数	先验知识	适用范围
L-BFGS	优化搜索	有目标	多次	白盒	特异攻击
Deep Fool	优化搜索	无目标	多次	白盒	特异攻击
UAP	优化搜索	无目标	多次	白盒	通用攻击
FGSM	特征构造	无目标	单次	白盒	特异攻击
BIM	特征构造	无目标	多次	白盒	特异攻击
LLC	特征构造	无目标	多次	白盒	特异攻击
JSMA	特征构造	有目标	多次	白盒	特异攻击
PBA	特征构造	有目标＆无目标	多次	黑盒	特异攻击
ATN	生成模型	有目标＆无目标	多次	白盒＆黑盒	特异攻击
AdvGAN	生成模型	有目标	多次	白盒	特异攻击

深度学习中对抗样本的构造及防御研究

Research on structure and defense of adversarial example in deep learning

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 52

相关文章 15

Metrics

推荐阅读 0

[1]	王滨, 陈靓, 钱亚冠, 郭艳凯, 邵琦琦, 王佳敏. 面向对抗样本攻击的移动目标防御[J]. 网络与信息安全学报, 2021, 7(1): 113-120.
[2]	杨路辉,白惠文,刘光杰,戴跃伟. 基于可分离卷积的轻量级恶意域名检测模型[J]. 网络与信息安全学报, 2020, 6(6): 112-120.
[3]	刘西蒙,谢乐辉,王耀鹏,李旭如. 深度学习中的对抗攻击与防御[J]. 网络与信息安全学报, 2020, 6(5): 36-53.
[4]	韩磊,刘吉强,王健,石波,和旭东. 高安全属性价值设备威胁态势量化评估方法[J]. 网络与信息安全学报, 2020, 6(5): 54-66.
[5]	杜思佳,于海宁,张宏莉. 基于深度学习的文本分类研究进展[J]. 网络与信息安全学报, 2020, 6(4): 1-13.
[6]	翟明芳,张兴明,赵博. 基于深度学习的加密恶意流量检测研究[J]. 网络与信息安全学报, 2020, 6(3): 66-77.
[7]	严飞,张铭伦,张立强. 基于边界值不变量的对抗样本检测方法[J]. 网络与信息安全学报, 2020, 6(1): 38-45.
[8]	王易东, 刘培顺, 王彬. 基于深度学习的系统日志异常检测研究[J]. 网络与信息安全学报, 2019, 5(5): 105-118.
[9]	尹赢,吉立新,黄瑞阳,杜立新. 网络表示学习的研究与发展[J]. 网络与信息安全学报, 2019, 5(2): 77-87.
[10]	李珍, 邹德清, 王泽丽, 金海. 面向源代码的软件漏洞静态检测综述[J]. 网络与信息安全学报, 2019, 5(1): 1-14.
[11]	宋蕾, 马春光, 段广晗. 机器学习安全及隐私保护研究进展[J]. 网络与信息安全学报, 2018, 4(8): 1-11.
[12]	燕昺昊,韩国栋. 基于深度循环神经网络和改进SMOTE算法的组合式入侵检测模型[J]. 网络与信息安全学报, 2018, 4(7): 48-59.
[13]	明拓思宇, 陈鸿昶. 文本摘要研究进展与趋势[J]. 网络与信息安全学报, 2018, 4(6): 1-10.
[14]	王宇龙,刘开元. 基于面部特征点运动的活体识别方法[J]. 网络与信息安全学报, 2018, 4(6): 36-44.
[15]	江玉朝,吉立新,高超,李邵梅. 面向Logo识别的合成数据生成方法研究[J]. 网络与信息安全学报, 2018, 4(5): 21-31.