基于半监督学习的人脸识别反欺骗方法研究

doi:10.11959/j.issn.2096-6652.202138

摘要/Abstract

摘要：

鉴别图像中的真伪人脸是一个长期具有挑战性的问题。当合成的伪造人脸十分逼真时，机器识别难分真假，甚至肉眼也难以区分。基于监督学习的真伪人脸识别建模往往需要大量的标签样本，模型的性能严重依赖样本的规模。提出一种基于半监督学习的人脸识别反欺骗方法，以减少对大量标签样本的依赖。该方法利用图像修复模型来学习人脸图像潜在的数据分布。在训练过程中，少量标签样本周期性地提供有监督信号来训练分类器，以区分真伪人脸。该方法可用于不同场景的伪造人脸，如基于摄像头拍摄的人脸或生成对抗网络生成的人脸。在NUAA 数据集和口罩遮挡人脸数据集 RMFD 上进行验证，实验结果表明，所提方法能够在不降低修复图像质量的情况下达到理想的分类精度。仅依靠少量带标签图像，所提方法比Improved-GAN方法和常用的半监督机器学习方法优势更明显，并优于支持向量机和卷积神经网络的监督学习方法。

关键词: 人脸反欺骗, 半监督学习, 生成对抗网络, 图像修复

Abstract:

It is a long-term challenge to identify the real and fake faces in the images.When the synthetic fake faces are very realistic, it is difficult for machines and even naked eyes to distinguish the real and fake ones.The supervised anti-spoofing method often requires a large number of labeled samples for a good performance.An anti-spoofing method of face recognition based on semi-supervised learning was proposed to reduce the dependence on massive labeled samples.The method adopted an image inpainting model to learn the data distribution of face images.During the training process, a few labeled samples periodically provided supervised signals to train the classifier to distinguish real faces from fake ones.The proposed method could be used for face anti-spoofing in different scenario, such as faces captured by cameras or generated by generative adversarial net.Accordingly, it was evaluated on the NUAA and RMFD datasets.Experiment results show that the proposed method can keep the quality of restored images, and achieve desirable classification accuracy.With a few labeled samples, the proposed method outperforms Improved-GAN and common semi-supervised methods, and surpasses supervised learning method based on support vector machine and convolutional neural network.

Key words: face anti-spoofing, semi-supervised learning, generative adversarial network, image inpainting

中图分类号:

TP183

李莉, 曾伟良, 黄永慧, 等. 基于半监督学习的人脸识别反欺骗方法研究[J]. 智能科学与技术学报, 2021, 3(3): 370-380.

Li LI, Weiliang ZENG, Yonghui HUANG, et al. Research on anti-spoofing method of face recognition based on semi-supervised learning[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 370-380.

图/表 15

图1

图2

表1

表2

图3

图4

图5

图6

图7

图8

表3

表4

表5

NUAA数据集监督学习的分类准确率"

方法	100张带标签图像	200张带标签图像	300张带标签图像	所有训练集
SVM	87.36%	89.38%	95.01%	99.80%
CNN	96.55%	98.65%	98.53%	99.96%
本文方法	$98 . 01 %$	$98 . 89 %$	$99 . 21 %$	-

表5

表6

RMFD监督学习的分类准确率"

方法	100 张带标签图像	200 张带标签图像	300 张带标签图像	所有训练集
SVM	53.95%	58.20%	62.50%	67.45%
CNN	51.35%	54.80%	59.36%	87.40%
本文方法	$69 . 27 %$	$74 . 55 %$	$83 . 05 %$	-

表6

表7

参考文献 41

[1]	LI J W , WANG Y H , TAN T N ,et al. Live face detection based on the analysis of Fourier spectra[C]// Proceedings of SPIE.[S.l.:s.n.], 2004: 296-303.
[2]	BOULKENAFET Z , KOMULAINEN J , HADID A . Face anti-spoofing based on color texture analysis[C]// Proceedings of 2015 IEEE International Conference on Image Processing. Piscataway:IEEE Press, 2015: 2636-2640.
[3]	CHOI J Y , PLATANIOTIS K N , RO Y M . Using colour local binary pattern features for face recognition[C]// Proceedings of 2010 IEEE International Conference on Image Processing. Piscataway:IEEE Press, 2010: 4541-4544.
[4]	BHARADWAJ S , DHAMECHA T I , VATSA M ,et al. Face anti-spoofing via motion magnification and multifeature videolet aggregation[R]. 2014.
[5]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence is entering the post deep-learning era[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 4-6.
[6]	LI L , FENG X Y , BOULKENAFET Z ,et al. An original face anti-spoofing approach using partial convolutional neural network[C]// Proceedings of 2016 6th International Conference on Image Processing Theory,Tools and Applications. Piscataway:IEEE Press, 2016: 1-6.
[7]	LIU Y J , JOURABLOO A , LIU X M . Learning deep models for face anti-spoofing:binary or auxiliary supervision[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 389-398.
[8]	PATEL K , HAN H , JAIN A K . Cross-database face antispoofing with robust feature representation[C]// Proceedings of 2016 Chinese Conference on Biometric Recognition. Heidelberg:Springer, 2016: 611-619.
[9]	RAGHAVENDRA R , BUSCH C . Novel presentation attack detection algorithm for face recognition system:application to 3D face mask attack[C]// Proceedings of 2014 IEEE International Conference on Image Processing. Piscataway:IEEE Press, 2014: 323-327.
[10]	FENG L T , PO L M , LI Y M ,et al. Integration of image quality and motion cues for face anti-spoofing:a neural network approach[J]. Journal of Visual Communication and Image Representation, 2016,38: 451-460.
[11]	GOODFELLOW I J , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial nets[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2014: 2672-2680.
[12]	SPRINGENBERG J T . Unsupervised and semi-supervised learning with categorical generative adversarial networks[J]. arXiv preprint,2015,arXiv:1511.06390.
[13]	SUTSKEVER I , JOZEFOWICZ R , GREGOR K ,et al. Towards principled unsupervised learning[J]. arXiv preprint,2015,arXiv:1511.06440.
[14]	ODENA A . Semi-supervised learning with generative adversarial networks[J]. arXiv preprint,2016,arXiv:1606.01583.
[15]	余翀 . 基于半监督生成对抗网络的三维重建云工作室[J]. 智能科学与技术学报, 2019,1(1): 70-82.
	YU C . Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 70-82.
[16]	TRIGUERO I , GARCíA S , HERRERA F . Self-labeled techniques for semi-supervised learning:taxonomy,software and empirical study[J]. Knowledge and Information Systems, 2015,42(2): 245-284.
[17]	WANG W , ZHOU Z H . A new analysis of co-training[C]// Proceedings of the 27th International Conference on Machine Learning.[S.l.:s.n], 2010.
[18]	DU J , LING C X , ZHOU Z H . When does cotraining work in real data?[J]. IEEE Transactions on Knowledge and Data Engineering, 2011,23(5): 788-799.
[19]	CHEN M , WEINBERGER K.Q , CHEN Y . Automatic feature decomposition for single view co-training[C]// Proceedings of the 27th International Conference on Machine Learning.[S.l.:s.n], 2011.
[20]	SHEIKHPOUR R , SARRAM M A , GHARAGHANI S ,et al. A survey on semi-supervised feature selection methods[J]. Pattern Recognition, 2017,64: 141-158.
[21]	GOLDBERG A B , ZHU X J , SINGH A ,et al. Multi-manifold semi-supervised learning[C]// Proceedings of the 12th International Conference on Artificial Intelligence and Statistics.[S.l.:s.n], 2009: 169-176.
[22]	ERHAN D , BENGIO Y , COURVILLE A ,et al. Why does unsupervised pre-training help deep learning[C]// Proceedings of the 13th International Conference on Artificial Intelligence and Statistics.[S.l.:s.n], 2010: 201-208.
[23]	王晓霞, 蒋伏松, 王宇 ,等. 基于ICD-10诊断编码的慢性病并发症聚类算法[J]. 大数据, 2018,4(3): 37-45.
	WANG X X , JIANG F S , WANG Y ,et al. Chronic disease complications clustering based on ICD-10 diagnoses code[J]. Big Data Research, 2018,4(3): 37-45.
[24]	LIU W , HE J F , CHANG S F . Large graph construction for scalable semi-supervised learning[C]// Proceedings of the 27th International Conference on Machine Learning.[S.l.:s.n], 2010: 679-686.
[25]	DHILLON P S , KEERTHI S S , BELLARE K ,et al. Deterministic annealing for semi-supervised structured output learning[C]// Proceedings of the 13th International Conference on Artificial Intelligence and Statistics.[S.l.:s.n], 2012: 299-307.
[26]	BREVE F , ZHAO L , QUILES M ,et al. Particle competition and cooperation in networks for semi-supervised learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2012,24(9): 1686-1698.
[27]	SALIMANS T , GOODFELLOW I , ZAREMBA W ,et al. Improved techniques for training GANs[C]// Proceedings of the 30th Conference on Neural Information Processing Systems.[S.l.:s.n], 2016: 2234-2242.
[28]	DAI Z H , YANG Z L , YANG F ,et al. Good semi-supervised learning that requires a bad GAN[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2017: 6510-6520.
[29]	PATHAK D , KR?HENBüHL P , DONAHUE J ,et al. Context encoders:feature learning by inpainting[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 2536-2544.
[30]	IIZUKA S , SIMO-SERRA E , ISHIKAWA H . Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 2017,36(4): 1-14.
[31]	LI Y J , LIU S F , YANG J M ,et al. Generative face completion[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 5892-5900.
[32]	KERZHNER Y , ELAD A , ROMANO Y . Image inpainting using pre-trained classification CNN[C]// Proceedings of 2018 International Conference on Sustainable Energy Engineering.[S.l.:s.n.], 2018: 1-9.
[33]	ALTINEL F , OZAY M , OKATANI T . Deep structured energy-based image inpainting[C]// Proceedings of 2018 24th International Conference on Pattern Recognition. Piscataway:IEEE Press, 2018: 423-428.
[34]	LAHIRI A , JAIN A , BISWAS P K ,et al. Improving consistency and correctness of sequence inpainting using semantically guided generative adversarial network[J]. arXiv preprint,2017,arXiv:1711.06106.
[35]	ZEILER M D . ADADELTA:an adaptive learning rate method[J]. arXiv preprint,2012,arXiv:1212.5701.
[36]	RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint,2015,arXiv:1511.06434.
[37]	YOSHUA B , OLIVIER D , ROUX N L . Label propagation and quadratic criterion[M]// Semi-supervised learning. Cambridge: The MIT Press, 2006: 192-216.
[38]	WANG F , ZHANG C S . Label propagation through linear neighborhoods[J]. IEEE Transactions on Knowledge and Data Engineering, 2008,20(1): 55-67.
[39]	HARTIGAN J A , WONG M A . Algorithm AS 136:a K-means clustering algorithm[J]. Applied Statistics, 1979,28(1): 100-108.
[40]	FENG H Z , KONG K Z , CHEN M H ,et al. SHOT-VAE:semi-supervised deep generative models with label-aware ELBO approximations[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2021: 7413-7421.
[41]	SAUNDERS C , STITSON M O , WESTON J ,et al. Support vector machine[J]. Computer Science, 2002,1(4): 1-28.

网络层类型	卷积核	步长	通道数量/个
卷积层	5×5	2×2	64
卷积层	5×5	2×2	128
卷积层	5×5	2×2	256
卷积层	5×5	2×2	512
卷积层	5×5	2×2	512
卷积层	5×5	2×2	512
全连接层	-	-	1 024

权重值	NUAA	RMFD
1	89.49%	86.95%
0.4	89.14%	87.05%
0.04	88.63%	86.65%
0.004	88.43%	87.80%
0.000 4	89.21%	86.35%
0.000 04	88.82%	86.30%

方法	100张	200张	300张
LS(RBF)	59.53%	59.53%	59.53%
LS(KNN)	71.90%	80.74%	86.68%
LP(RBF)	50.00%	50.00%	50.00%
LP(KNN)	59.65%	59.89%	60.76%
K-means+SVM	92.91%	94.54%	97.82%
Improved-GAN	94.49%	96.07%	96.27%
SHOT-VAE	95.88%	96.12%	98.80%
本文方法	98.01%	98.89%	99.21%

方法	100张	200张	300张
LS(RBF)	49.34%	49.34%	49.34%
LS(KNN)	50.27%	51.80%	53.30%
LP(RBF)	40.47%	40.47%	40.47%
LP(KNN)	50.00%	51.40%	51.60%
K-means+SVM	60.52%	61.08%	62.78%
Improved-GAN	67.14%	69.75%	73.19%
SHOT-VAE	65.05%	67.10%	69.96%
本文方法	62.27%	74.55%	83.05%

	100张	200张	300张
Improved-GAN	61.09%	62.34%	63.96%
SHOT-VAE	58.90%	59.27%	64.15%
本文方法	60.47%	62.96%	64.57%