基于双层注意力的Deepfake换脸检测

doi:10.11959/j.issn.2096-109x.2021032

Abstract

Abstract:

In view of the existing Deepfake detection algorithms, such problems as low accuracy and poor interpretability are common.A neural network model combining the double attention was proposed, which used channel attention to capture the abnormal features of false faces and combined the location of spatial attention to focus the abnormal features.To fully learn the contextual semantic information of the abnormal part of the false face, so as to improve the effectiveness and accuracy of face changing detection.In addition, the decision-making area of real and fake faces was shown effectively in the form of thermal diagram, which provided a certain degree of explanation for the face exchange detection model.Experiments on FaceForensics ++ open source data set show that the detection accuracy of proposed method is superior to MesoInception, Capsule-Forensics and XceptionNet.

Key words: Deepfake, face swap detection, fake face detection, attention

CLC Number:

TP393

Xiaojuan GONG, Tianqiang HUANG, Bin WENG, Feng YE, Chao XU, Lijun YOU. Deepfake swapped face detection based on double attention[J]. Chinese Journal of Network and Information Security, 2021, 7(2): 151-160.

Figures/Tables 12

网络	Deepfake	FaceSwap	Face2face
XceptionNet^[2]	94.81%	92.76%	88.00%
XceptionNet-DA	97.96%	97.32%	96.36%
ResNet18^[14]	55.43%	57.29%	53.36%
DANet-18	96.04%	93.57%	96.07%
ResNet50^[14]	56.21%	63.89%	61.25%
DANet-50	$98 . 61 %$	$98 . 86 %$	$98 . 11 %$

网络	先放置空间注意力后放置通道注意力	先放置通道注意力后放置空间注意力
Deepfake	98.57%	$98 . 61 %$
FaceSwap	92.00%	$98 . 86 %$
Face2face	84.14%	$98 . 11 %$

网络	Deepfake	FaceSwap	Face2face
MesoInception^[18]	90.07%	90.89%	96.00%
Capsule-Forensics^[7]	92.17%	92.79%	90.00%
XceptionNet^[2]	94.81%	92.76%	88.00%
DANet	$98 . 61 %$	$98 . 86 %$	$98 . 11 %$

References 27

[1]	THIES J , ZOLLHOFER M , STAMMINGER M ,et al. Demo of Face2Face:real-time face capture and reenactment of RGB videos[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016(1): 2387-2395.
[2]	R?SSLER A , COZZOLINOD , VERDOLIVAL , ,et al. FaceForensics++:learning to detect manipulated facial images[C]// International Conference on Computer Vision(ICCV). 2019: 1-11.
[3]	LI Y , CHANG M , FARID H ,et al. In ictu.oculi:exposing AI generated fake face videos by detecting eye blinking[C]// International Workshop on Information Forensics and Security(WIFS). 2018: 1-7.
[4]	COZZOLINO D , VERDOLIVA L . Noiseprint:a CNN-based camera model fingerprint[J]. IEEE Transactions on Information Forensics and Security, 2020(15): 144-159.
[5]	DONG H , LIU F , STEHOUWER J ,et al. On the detection of digital face manipulation[J]. arXiv:1910.01717[astro-ph.CO].
[6]	ZHOU C , WU M , LAM S . SSA-CNN:semantic self-attention CNN for pedestrian detection[J]. arXiv:1902.09080v1[astro-ph.CO].
[7]	NGUYEN H , YAMAGISHI J , ECHIZEN I . Use of a capsule network to detect fake images and videos[J]. arXiv:1910.12467v2[astro-ph.CO].
[8]	SELVARAJU M , COGSWELL M , DAS A . Grad-CAM:visual explanations from deep networks via gradient-based localization[C]// International Conference on Computer Vision (ICCV). 2017: 618-626.
[9]	YANG X , LI Y , LYU S . Exposing deep fakes using inconsistent head poses[C]// 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). 2019: 8261-8265.
[10]	MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose Deepfakes and face manipulations[C]// 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). 2019: 83-92.
[11]	NATARAJ L , MOHAMMED T M , MANJUNATH B S ,et al. Detecting GAN generated fake images using co-occurrence matrices[C]// Media Watermarking,Security,and Forensics. 2019.
[12]	LI Y , LYU S . Exposing Deepfake videos by detecting face warping artifacts[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 46-52.
[13]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[C]// 3rd International Conference on Learning Representations (ICLR). 2015.
[14]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). 2016: 770-778.
[15]	GUERA D , DELP E . Deepfake video detection using recurrent neural networks[C]// 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 2018: 1-6.
[16]	DONAHUE J , HENDRICKS L , GUADARRAMA S ,et al. Long-term recurrent convolutional networks for visual recognition and description[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015: 2625-2634.
[17]	AMERINI I , GALTERI L , CALDELLI R ,et al. Deepfake video detection through optical flow based CNN[C]// International Conference on Computer Vision Workshops (ICCVW). 2019: 1205-1207.
[18]	AFCHAR D , NOZICK V , YAMAGISHI J ,et al. Mesonet:a compact facial video forgery detection network[C]// 2018 IEEE International Workshop on Information Forensics and Security (WIFS). 2018: 1-7.
[19]	FERNANDES S , RAJ S , ORTIZ E ,et al. Predicting heart rate variations of Deepfake videos using neural ODE[C]// International Conference on Computer Vision Workshops(ICCVW). 2019: 1721-1729.
[20]	SABIR E , CHENG J , JAISWAL A ,et al. Recurrent convolutional strategies for face manipulation detection in videos[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 80-87.
[21]	NGUYEN H , FUMING F , YAMAGISHI J ,et al. Multitask learning for detecting and segmenting manipulated facial images and videos[J]. arXiv:abs/1906.06876[astro-ph.CO].
[22]	CHOLLET F , . Xception:deep learning with depth wise separable convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 1800-1807.
[23]	LI H , CHEN G , LI G ,et al. Motion guided attention for video salient object detection[C]// International Conference on Computer Vision (ICCV). 2019: 7273-7282.
[24]	LIU H , JIANG B , XIAO Y ,et al. Coherent semantic attention for image inpainting[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 4169-4178.
[25]	FU J , LIU J , TIAN H ,et al. Dual attention network for scene segmentation[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 3146-3154.
[26]	HU J , SHEN LI , SUN G . Squeeze-and-excitation networks[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018: 7132-7141.
[27]	WOO S , PARK J , LEE J ,et al. CBAM:convolutional block attention module[C]// The European Conference on Computer Vision(ECCV). 2018: 3-19.

Metrics

Recommended 0

No Suggested Reading articles found!

FaceForensics++数据集		Deepfake	FaceSwap	Face2face
训练集	Original	72 000	72 000	72 000
训练集	Fake	72 000	72 000	72 000
验证集	Original	1 400	1 400	1 400
验证集	Fake	1 400	1 400	1 400
测试集	Original	1 400	1 400	1 400
测试集	Fake	1 400	1 400	1 400

标注	含义	标注	含义
Ori	真脸	Ori_cam	真脸的热力图
Deep	Deepfake假脸	Deep_cam	Deepfake假脸的热力图
Swap	FaceSwap假脸	Swap_cam	FaceSwap假脸的热力图
F2f	Face2face假脸	F2f_cam	Face2face假脸的热力图

Deepfake swapped face detection based on double attention

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 27

Related Articles 12

Metrics

Recommended 0

[1]	Guanyun FENG, Cai FU, Jianqiang LYU, Lansheng HAN. Insider threat detection based on operational attention and data augmentation [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 102-112.
[2]	Chuntao ZHU, Chengxi YIN, Bolin ZHANG, Qilin YIN, Wei LU. Forgery face detection method based on multi-domain temporal features mining [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 123-134.
[3]	Jingwen LI, Yawen LI. Application and risk response of deep synthesis technology [J]. Chinese Journal of Network and Information Security, 2023, 9(2): 184-190.
[4]	Wenxuan WU, Wenbo ZHOU, Weiming ZHANG, Nenghai YU. Deepfake detection method based on patch-wise lighting inconsistency [J]. Chinese Journal of Network and Information Security, 2023, 9(1): 167-177.
[5]	Dengyong ZHANG, Huang WEN, Feng LI, Peng CAO, Lingyun XIANG, Gaobo YANG, Xiangling DING. Image inpainting forensics method based on dual branch network [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 110-122.
[6]	Jiaying LIN, Wenbo ZHOU, Weiming ZHANG, Nenghai YU. Lip forgery detection via spatial-frequency domain combination [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 146-155.
[7]	Fan GAO, Jian WANG, Jiqiang LIU. Research on link detection technology based on dynamic browser fingerprint [J]. Chinese Journal of Network and Information Security, 2022, 8(4): 144-156.
[8]	Xiangdong HU, Zhengguo TIAN. Methods of security situation prediction for industrial internet fused attention mechanism and BSRU [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 41-51.
[9]	Pengcheng WANG, Haibin ZHENG, Jianfei ZOU, Ling PANG, Hu LI, Jinyin CHEN. Robustness evaluation of commercial liveness detection platform [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 180-189.
[10]	Jian JIA,Linfeng LIU,Jiagao WU. Charging pile recommendation method for idle electric taxis based on recurrent neural network [J]. Chinese Journal of Network and Information Security, 2020, 6(6): 152-163.
[11]	Qiang QU,Hongtao YU,Ruiyang HUANG. Attention-based approach of detecting spam in social networks [J]. Chinese Journal of Network and Information Security, 2020, 6(1): 54-61.
[12]	Xiaobin ZHANG, Fucai CHEN, Ruiyang HUANG. Relation extraction based on CNN and Bi-LSTM [J]. Chinese Journal of Network and Information Security, 2018, 4(9): 44-51.