针对车牌识别系统的双重对抗攻击

doi:10.11959/j.issn.2096-109x.2023034

网络与信息安全学报 ›› 2023, Vol. 9 ›› Issue (3): 16-27.doi: 10.11959/j.issn.2096-109x.2023034

针对车牌识别系统的双重对抗攻击

陈先意¹, 顾军¹, 颜凯¹, 江栋¹, 许林峰¹, 付章杰¹^,²

¹ 南京信息工程大学数字取证教育部工程研究中心，江苏南京 210044
² 综合业务网理论及关键技术国家重点实验室（西安电子科技大学），陕西西安 710126

修回日期:2023-04-18 出版日期:2023-06-25 发布日期:2023-06-01
作者简介:陈先意（1983- ），男，湖北恩施人，南京信息工程大学副教授，主要研究方向为区块链安全、大数据安全及人工智能安全
顾军（1996- ），男，江苏盐城人，南京信息工程大学硕士生，主要研究方向为对抗样本、人工智能安全
颜凯（1998- ），男，江苏淮安人，南京信息工程大学硕士生，主要研究方向为对抗攻击
江栋（1998- ），男，四川成都人，南京信息工程大学硕士生，主要研究方向为人工智能安全、对抗样本
许林峰（1998- ），男，江苏南通人，南京信息工程大学硕士生，主要研究方向为信息安全、对抗样本
付章杰（1983- ），男，河南南阳人，南京信息工程大学教授、博士生导师，主要研究方向为区块链安全、数字取证、人工智能安全
基金资助:
国家重点研发计划(2021YFB2700900);国家自然科学基金(62172232);国家自然科学基金(62172233);江苏省杰出青年基金(BK20200039);江苏省大气环境与装备技术协同创新中心基金

Double adversarial attack against license plate recognition system

Xianyi CHEN¹, Jun GU¹, Kai YAN¹, Dong JIANG¹, Linfeng XU¹, Zhangjie FU¹^,²

¹ Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China
² The State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an 710126, China

Revised:2023-04-18 Online:2023-06-25 Published:2023-06-01
Supported by:
The National Key R＆D Program of China(2021YFB2700900);The National Natural Science Foundation of China(62172232);The National Natural Science Foundation of China(62172233);The Jiangsu Basic Research Programs-Natural Science Foundation(BK20200039);The Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET) Fund

摘要/Abstract

摘要：

使用深度神经网络的人工智能系统面对对抗样本的攻击表现出极大的脆弱性。因此，在基于深度神经网络的车牌识别系统（LPR，license plate recognition）场景下，提出针对车牌识别系统的双重对抗攻击（DAA，double adversarial attack）方法。通过添加对抗补丁到车牌的图案位置，使LPR的目标检测子系统不能检测到车牌类；通过在车牌号上添加不规则的单连通区域扰动斑点以模拟自然形成的铁锈、污渍等，使 LPR 的车牌号识别子系统产生误识别。针对车牌研究设计不同形状的对抗补丁、不同颜色的对抗斑点，以此产生对抗车牌，并迁移到物理世界中。实验结果表明，设计出的对抗样本既不会被人眼所察觉，也能够欺骗车牌识别系统，如EasyPR，在物理世界中的攻击成功率能够达到99%。关于LPR的对抗攻击及深度学习的脆弱性研究，对提高车牌识别模型的鲁棒性具有积极促进意义。

关键词: 车牌识别, 对抗补丁, 对抗斑点, 对抗攻击

Abstract:

Recent studies have revealed that deep neural networks (DNN) used in artificial intelligence systems are highly vulnerable to adversarial sample-based attacks.To address this issue, a dual adversarial attack method was proposed for license plate recognition (LPR) systems in a DNN-based scenario.It was demonstrated that an adversarial patch added to the pattern location of the license plate can render the target detection subsystem of the LPR system unable to detect the license plate class.Additionally, the natural rust and stains were simulated by adding irregular single-connected area random points to the license plate image, which results in the misrecognition of the license plate number.or the license plate research, different shapes of adversarial patches and different colors of adversarial patches are designed as a way to generate adversarial license plates and migrate them to the physical world.Experimental results show that the designed adversarial samples are undetectable by the human eye and can deceive the license plate recognition system, such as EasyPR.The success rate of the attack in the physical world can reach 99%.The study sheds light on the vulnerability of deep learning and the adversarial attack of LPR, and offers a positive contribution toward improving the robustness of license plate recognition models.

Key words: license plate recognition, adversarial patch, adversarial spot, adversarial attack

中图分类号:

TP393

陈先意, 顾军, 颜凯, 江栋, 许林峰, 付章杰. 针对车牌识别系统的双重对抗攻击[J]. 网络与信息安全学报, 2023, 9(3): 16-27.

Xianyi CHEN, Jun GU, Kai YAN, Dong JIANG, Linfeng XU, Zhangjie FU. Double adversarial attack against license plate recognition system[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 16-27.

图/表 14

图1

图2

图3

表1

图4

表2

图5

图6

表3

图7

图8

图9

表4

表5

参考文献 35

[1]	SZEGEDY C , ZAREMBA W , SUTSKEVER I ,et al. Intriguing properties of neural networks[J]. arXiv Preprint arXiv:13126199, 2013.
[2]	潘文雯, 王新宇, 宋明黎 ,等. 对抗样本生成技术综述[J]. 软件学报, 2020,31(1): 67-81.
	PAN W W , WANG X Y , SONG M L ,et al. Survey on generating adversarial examples[J]. Journal of Software, 2020,31(1): 67-81.
[3]	马钰锡, 张全新, 谭毓安 ,等. 面向智能攻击的行为预测研究[J]. 软件学报, 2020,32(5): 1526-1546.
	MA Y Xi , ZHANG Q X , TAN Y A ,et al. Research on behavior-prediction for intelligent attacks[J]. Journal of Software, 2020,32(5): 1526-1546.
[4]	陈晋音, 沈诗婧, 苏蒙蒙 ,等. 车牌识别系统的黑盒对抗攻击[J]. 自动化学报, 2021,47(1): 121-135.
	CHEN J Y , SHEN S J , SU M M ,et al. Black-box adversarial attack on license plate recognition system[J]. Acta Automatica Sinica, 2021,47(1): 121-135.
[5]	BIGGIO B , CORONA I , MAIORCA D ,et al. Evasion attacks against machine learning at test time[C]// Machine Learning and Knowledge Discovery in Databases:European Conference,ECML PKDD 2013,Prague,Czech Republic, 2013: 387-402.
[6]	GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples[J]. arXiv Preprint arXiv:14126572, 2014.
[7]	MOOSAVI-DEZFOOLI S M , FAWZI A , FROSSARD P . Deepfool:a simple and accurate method to fool deep neural networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition,Las Vegas,America, 2016: 2574-2582.
[8]	CARLINI N , WAGNER D . Towards evaluating the robustness of neural networks[C]// 2017 ieee symposium on security and privacy (sp),San Jose,CA,USA, 2017: 39-57.
[9]	BROWN T B , MANé D , ROY A ,et al. Adversarial patch[J]. arXiv Preprint arXiv:171209665, 2017.
[10]	EYKHOLT K , EVTIMOV I , FERNANDES E ,et al. Robust physical-world attacks on deep learning visual classification[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 1625-1634.
[11]	ATHALYE A , ENGSTROM L , ILYAS A ,et al. Synthesizing robust adversarial examples[C]// International Conference on Machine Learning. 2018: 284-293.
[12]	MOOSAVI-DEZFOOLI S M , FAWZI A , FAWZI O ,et al. Universal adversarial perturbations[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1765-1773.
[13]	TSAI T , YANG K , HO T Y ,et al. Robust adversarial objects against deep learning models[C]// Proceedings of the AAAI Conference on Artificial Intelligence,Hilton New York Midtown, 2020: 954-962.
[14]	CHEN S T , CORNELIUS C , MARTIN J ,et al. Shapeshifter:Robust physical adversarial attack on faster R-CNN object detector[C]// Machine Learning and Knowledge Discovery in Databases:European Conference,ECML PKDD 2018,Dublin,Ireland, 2018: 52-68.
[15]	REN S , HE K , GIRSHICK R ,et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015,28.
[16]	REDMON J , FARHADI A . YOLO9000:better,faster,stronger[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 7263-7271.
[17]	THYS S , VAN R W , GOEDEMé T . Fooling automated surveillance cameras:adversarial patches to attack person detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2019.
[18]	XU K , ZHANG G , LIU S ,et al. Adversarial t-shirt! evading person detectors in a physical world[C]// Computer Vision–ECCV 2020:16th European Conference. 2020: 665-681.
[19]	KAZIAKHMEDOV E , KIREEV K , MELNIKOV G ,et al. Real-world attack on MTCNN face detection system[C]// 2019 International Multi-Conference on Engineering,Computer and Information Sciences (SIBIRCON). 2019: 422-427.
[20]	HUANG L , GAO C , ZHOU Y ,et al. Universal physical camouflage attacks on object detectors[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 720-729.
[21]	ZHU X , LI X , LI J ,et al. Fooling thermal infrared pedestrian detectors in real world using small bulbs[C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2021: 3616-3624.
[22]	HU Y C T , KUNG B H , TAN D S ,et al. Naturalistic physical adversarial patch for object detectors[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 7848-7857.
[23]	HU Z , HUANG S , ZHU X ,et al. Adversarial texture for fooling person detectors in the physical world[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,New Orleans Ernest N. 2022: 13307-13316.
[24]	WANG J , LIU A , YIN Z ,et al. Dual attention suppression attack:Generate adversarial camouflage in physical world[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 8565-8574.
[25]	DU A , CHEN B , CHIN T J ,et al. Physical adversarial attacks on an aerial imagery object detector[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,New Orleans Ernest N. 2022: 1796-1806.
[26]	WANG D , JIANG T , SUN J ,et al. Fca:Learning a 3d full-coverage vehicle camouflage for multi-view physical adversarial attack[C]// Proceedings of the AAAI Conference on Artificial Intelligence. , 2022: 2414-2422.
[27]	REDMON J , FARHADI A . Yolov3:An incremental improvement[J]. arXiv Preprint arXiv:180402767, 2018.
[28]	SURYANTO N , KIM Y , KANG H ,et al. Dta:Physical camouflage attacks using differentiable transformation network[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 15305-15314.
[29]	BOCHKOVSKIY A , WANG C Y , LIAO H Y M . Yolov4:Optimal speed and accuracy of object detection[J]. arXiv Preprint arXiv:200410934, 2020.
[30]	PURKAIT P , ZHAO C , ZACH C . SPP-Net:Deep absolute pose regression with synthetic views[J]. arXiv Preprint arXiv:171203452, 2017.
[31]	LIU S , QI L , QIN H ,et al. Path aggregation network for instance segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8759-8768.
[32]	PARISI R , DI CLAUDIO E D , LUCARELLI G ,et al. Car plate recognition by neural networks and image processing[C]// 1998 IEEE International Symposium on Circuits and Systems (ISCAS). 1998: 195-198.
[33]	QIAN Y , MA D , WANG B ,et al. Spot evasion attacks:Adversarial examples for license plate recognition systems with convolutional neural networks[J]. Computers ＆ Security, 2020,95:101826.
[34]	KWON H , BAEK J W . Adv-plate attack:adversarially perturbed plate for license plate recognition system[J]. Journal of Sensors, 2021,2021: 1-10.
[35]	WANG C Y , LIAO H Y M , WU Y H ,et al. CSPNet:A new backbone that can enhance learning capability of CNN[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops: 390-391.

模型名称	训练识别准确率	测试识别准确率
HyperLPR	95.7%	95.3%
EasyPR	89.4%	87.6%
OpenALPR	97.6%	96.8%
PaddleOCR	90.2%	87.9%

补丁形状	攻击迭代次数
补丁形状	200	400	600	800	1 000
圆形补丁	14.25%	41.55%	72.24%	86.92%	91.85%
椭圆形补丁	12.37%	33.86%	61.23%	75.71%	84.13%
矩形补丁	7.32%	24.21%	45.69%	66.73%	75.11%

输入类别	攻击成功率
输入类别	Yang等	本文
数字世界	98.7%	99.9%
物理世界	98.2%	98.4%

模型	成功率
HyperLPR	98%
EasyPR	99%
PaddleOCR	94%
OpenALPR	98%

原始车牌	攻击场景1	攻击场景2	攻击场景3

	HyperLPRNone	HyperLPRDZVJ49	HyperLPRDZV649
	EasyPRNone	EasyPRNone	EasyPRDZV49
DZVG49	PaddleOCRDZG49	PaddleOCRD2VG9	PaddleOCRDZVG49
	OpenALPRNone	OpenALPRNone	OpenALPRDZV49

针对车牌识别系统的双重对抗攻击

Double adversarial attack against license plate recognition system

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 35

相关文章 4

Metrics

推荐阅读 0

[1]	陈晋音, 李荣昌, 黄国瀚, 刘涛, 郑海斌, 程瑶. 纵向联邦学习方法及其隐私和安全综述[J]. 网络与信息安全学报, 2023, 9(2): 1-20.
[2]	张宇, 李海良. 基于RSA的图像可识别对抗攻击方法[J]. 网络与信息安全学报, 2021, 7(5): 40-48.
[3]	陈晋音, 张敦杰, 黄国瀚, 林翔, 鲍亮. 面向图神经网络的对抗攻击与防御综述[J]. 网络与信息安全学报, 2021, 7(3): 1-28.
[4]	刘西蒙,谢乐辉,王耀鹏,李旭如. 深度学习中的对抗攻击与防御[J]. 网络与信息安全学报, 2020, 6(5): 36-53.