面向人脸识别的口罩区域修复算法

doi:10.11959/j.issn.1000-0801.2021193

电信科学 ›› 2021, Vol. 37 ›› Issue (8): 66-76.doi: 10.11959/j.issn.1000-0801.2021193

面向人脸识别的口罩区域修复算法

李悦¹, 钱亚冠¹, 关晓惠², 李蔚¹, 王滨³, 顾钊铨⁴

¹ 浙江科技学院大数据学院，浙江杭州 310023
² 浙江水利水电学院信息工程与艺术设计学院，浙江杭州 310027
³ 杭州海康威视网络与信息安全实验室，浙江杭州 310051
⁴ 广州大学网络空间先进技术研究院，广东广州 510006

修回日期:2021-08-11 出版日期:2021-08-20 发布日期:2021-08-01
作者简介:李悦（1996− ），女，浙江科技学院硕士生，主要研究方向为深度学习、计算机视觉
钱亚冠（1976− ），男，浙江科技学院副教授，主要研究方向为深度学习、人工智能安全和大数据处理
关晓惠（1977− ），女，浙江水利水电学院副教授，主要研究方向为数字图像处理与模式识别
李蔚（1978− ），女，浙江科技学院副教授，主要研究方向为科学计算、模式识别和计算机视觉中的机器学习问题
王滨（1977− ），男，杭州海康威视网络与信息安全实验室研究员，主要研究方向为物联网安全、人工智能安全和网络安全
顾钊铨（1989− ），男，广州大学网络空间先进技术研究院教授，主要研究方向为无线网络、分布式计算、大数据分析和人工智能安全
基金资助:
国家重点研发计划项目(2018YFB2100400);国家自然科学基金资助项目(61902082);浙江省公益技术应用研究项目(LGF20F020007)

Region inpainting algorithm of mouth-muffles for facial recognition

Yue LI¹, Yaguan QIAN¹, Xiaohui GUAN², Wei LI¹, Bin WANG³, Zhaoquan GU⁴

¹ School of Big Data Science, Zhejiang University of Science and Technology, Hangzhou 310023, China
² College of Information Engineering ＆Art Design, Zhejiang University of Water Resources and Electric Power, Hangzhou 310027, China
³ Network and Information Security Laboratory of Hangzhou Hikvision Digital Technology Co., Ltd., Hangzhou 310051, China
⁴ Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou 510006, China

Revised:2021-08-11 Online:2021-08-20 Published:2021-08-01
Supported by:
The National Key R＆D Program of China(2018YFB2100400);The National Natural Science Foundation of China(61902082);Applied Research Project of Public Welfare Technology of Zhejiang Province of China(LGF20F020007)

摘要/Abstract

摘要：

遮挡下的人脸识别一直是现实场景中的一个难题。特别是新冠肺炎疫情爆发后，在机场、车站等需要鉴别入场人员身份信息的场所，口罩遮挡使得可供识别的面部特征大幅减少，原有的人脸识别算法准确率随之下降。对去除口罩遮挡进行了研究，提出了一个新的框架修复人脸，利用边缘生成网络还原遮挡区域的边缘，在此基础上再利用区域填充网络恢复被遮挡的人脸，同时保留身份信息。为提升模型的性能，提出空间加权对抗损失和身份一致性损失训练上述网络，并利用关键点信息，构建了两个戴口罩的人脸数据集。实验结果表明，恢复被口罩遮挡的人脸的图像使人脸识别算法 ArcFace 的准确率达到 98.39%，比直接采用ArcFace识别遮挡人脸提升了4.13%的准确率。

关键词: 口罩遮挡, 人脸识别, 边缘生成网络, 区域填充网络

Abstract:

Facial recognition under occlusion is a well-known difficult problem in real scenes.Especially after the outbreak of COVID-19, in airports, stations and other places that need to verify the identity of visitors, the mouth-muffle occlusion greatly reduces the facial features that can be important for identification, and the accuracy of face recognition algorithm decreases.Face de-occlusion was studied, and a novel framework was proposed to restore face, which used the edge generation network to generate edge maps.On this basis, the occluded region was restored by the region completion network while preserving identity information.In order to improve the performance, the spatial weighted adversarial loss and identity-preserving loss were introduced to train the above two networks.Then two face datasets with mouth-muffles were constructed by face landmarks.The experimental results show that the accuracy of facial recognition algorithm ArcFace on the face dataset restored by the proposed model was 98.39%, which was 4.13% higher than that of directly using ArcFace.

Key words: mouth-muffle occlusions, facial recognition, edge generation network, region completion network

中图分类号:

TP393

李悦, 钱亚冠, 关晓惠, 李蔚, 王滨, 顾钊铨. 面向人脸识别的口罩区域修复算法[J]. 电信科学, 2021, 37(8): 66-76.

Yue LI, Yaguan QIAN, Xiaohui GUAN, Wei LI, Bin WANG, Zhaoquan GU. Region inpainting algorithm of mouth-muffles for facial recognition[J]. Telecommunications Science, 2021, 37(8): 66-76.

图/表 17

图1

图2

图3

图4

图5

图6

图7

图8

图9

表1

图10

表2

表3

图11

图12

表4

表5

参考文献 23

[1]	TAIGMAN Y , YANG M , RANZATO M ,et al. DeepFace:closing the gap to human-level performance in face verification[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 1701-1708.
[2]	HUANG G B , MATTAR M , BERG T ,et al. Labeled faces in the wild:a database forStudying face recognition in unconstrained environments[EB]. 2008.
[3]	DENG J K , GUO J , XUE N N ,et al. ArcFace:additive angular margin loss for deep face recognition[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 4685-4694.
[4]	DUAN Q Y , ZHANG L . BoostGAN for occlusive profile face frontalization and recognition[EB]. 2019.
[5]	ZHAO F , FENG J S , ZHAO J ,et al. Robust LSTM- autoencoders for face de-occlusion in the wild[J]. IEEE Transactions on Image Processing, 2018,27(2): 778-790.
[6]	ZHAO Y J , CHEN W K , XING J ,et al. Identity preserving face completion for large ocular region occlusion[EB]. 2018.
[7]	WANG M , HU Z P , SUN Z ,et al. Varying face occlusion detection and iterative recovery for face recognition[J]. Journal of Electronic Imaging, 2017(26):033009.
[8]	ZHANG J , KAN M N , SHAN S G ,et al. Occlusion-free face alignment:deep regression networks coupled with de-corrupt AutoEncoders[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016: 3428-3437.
[9]	YUAN X W , PARK I K . Face de-occlusion using 3D morphable model and generative adversarial network[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2019: 10061-10070.
[10]	GOODFELLOW I , POUGET-ABADIE J ,, MIRZA M , et al . Generative adversarial nets[C]// Proceedings of the Advances in Neural Information Processing Systems. New York:IncPress, 2014: 2672-2680.
[11]	HE Z L , ZUO W M , KAN M N ,et al. AttGAN:facial attribute editing by only changing what you want[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2019,28(11): 5464-5478.
[12]	IIZUKA S , SIMO-SERRA E ,, ISHIKAWA H . Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 2017,36(4): 1-14.
[13]	LI Y J , LIU S F , YANG J M ,et al. Generative face completion[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 5892-5900.
[14]	GE S , LI C , ZHAO S ,et al. Occluded face recognition in the wild by identity-diversity inpainting[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020,30(10): 3387-3397.
[15]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. arXiv:1411.1784, 2014.
[16]	CANNY J . A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986,8(6): 679-698.
[17]	ZHANG K P , ZHANG Z P , LI Z F ,et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016,23(10): 1499-1503.
[18]	JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution[M]// Computer Vision –ECCV 2016. Cham: Springer International Publishing, 2016: 694-711.
[19]	GATYS L A , ECKER A S , BETHGE M . Image style transfer using convolutional neural networks[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016: 2414-2423.
[20]	BARTH N . The gramian and k-volume in n-space:some classical results in linear algebra[J]. Journal of Young Investigators, 1999,2(1): 1-4.
[21]	YI D , LEI Z , LIAO S ,et al. Learning face representation from scratch[J]. arXiv:1411.7923, 2014.
[22]	ISOLA P , ZHU J Y , ZHOU T H ,et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 5967-5976.
[23]	MAATEN L , HINTON G . Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008,9(3): 2579-2605.

对比项	CASIA-WebFace mask			LFW mask
对比项	EFC_roc	EFC_mask	ID-EFC	EFC_roc	EFC_mask	ID-EFC
SSIM	89.79%	94.38%	95.43%	94.41%	96.12%	97.57%
PSNR	26.58%	29.03%	30.20%	29.05%	30.90%	31.98%

对比项	CASIA-WebFace mask		LFW mask
对比项	EFC_mask	EFC_mask+L_adv	EFC_mask	EFC_mask+L_adv
SSIM	94.38%	95.20%	96.12%	97.43%
PSNR	29.03%	29.98%	30.90%	31.94%

数据集	EFC_roc	EFC_mask	EFC_mask+L_adv	EFC_mask+L_id	EFC_mask+L_adv+L_id	ArcFace
LFW	—	—	—	—	—	99.82%
LFW mask	93.87%	97.98%	98.06%	98.14%	98.39%	94.26%

口罩类型（遮挡面积）	黑色（40%）	白色（39%）	蓝色（37%）
人脸识别准确率	91.23%	95.77%	93.18%

遮挡面积		口罩颜色
遮挡面积	黑色	白色	蓝色
30%	94.21%	97.13%	95.44%
35%	92.61%	96.24%	93.68%
40%	91.23%	95.56%	92.72%

面向人脸识别的口罩区域修复算法

Region inpainting algorithm of mouth-muffles for facial recognition

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 23

相关文章 4

Metrics

推荐阅读 0

[1]	叶学义, 钱丁炜, 应娜, 王涛. 基于吸引模式的局部二阶梯度轮廓人脸识别算法[J]. 电信科学, 2021, 37(7): 96-106.
[2]	葛凯强, 陈铁明. 人机交互安全攻防综述[J]. 电信科学, 2019, 35(10): 100-116.
[3]	唐彪,金炜,符冉迪,龚飞. 多稀疏表示分类器决策融合的人脸识别[J]. 电信科学, 2018, 34(4): 31-40.
[4]	龚飞,金炜,朱珂晴,符冉迪,曹燕. 采用双字典协作稀疏表示的光照及表情顽健人脸识别[J]. 电信科学, 2017, 33(3): 52-58.