Chinese Journal of Network and Information Security ›› 2022, Vol. 8 ›› Issue (6): 146-155.doi: 10.11959/j.issn.2096-109x.2022075
• Papers and Reports • Previous Articles Next Articles
Jiaying LIN1,2, Wenbo ZHOU1,2, Weiming ZHANG1,2, Nenghai YU1,2
Revised:
2022-07-09
Online:
2022-12-15
Published:
2023-01-16
Supported by:
CLC Number:
Jiaying LIN, Wenbo ZHOU, Weiming ZHANG, Nenghai YU. Lip forgery detection via spatial-frequency domain combination[J]. Chinese Journal of Network and Information Security, 2022, 8(6): 146-155.
"
数据集 | Steg.Features[ | Cozzolino et al[ | Rahmouni et al[ | Bayer & Stamm[ | MesoNet[ | XceptionNet[ | 本文方法 | |||||||||||||
ACC | AUC | ACC | AUC | ACC | AUC | ACC | AUC | ACC | AUC | ACC | AUC | ACC | AUC | |||||||
C23 | 70.97% | — | 78.45% | — | 79.08% | — | 82.97% | — | 83.10% | — | 92.39% | 94.86% | 92.88% | 95.07% | ||||||
C40 | 55.98% | — | 58.69% | — | 61.18% | — | 66.84% | — | 70.47% | — | 80.32% | 81.76% | 81.50% | 82.81% |
"
检测方法 | ACC | |||
DeepFakes | Face2Face | FaceSwap | Neural Textures | |
Steg.Features[ | 73.64% | 73.72% | 68.93% | 63.33% |
Cozzolino et al[ | 85.45% | 67.88% | 73.79% | 78.00% |
Rahmouni et al[ | 85.45% | 64.23% | 56.31% | 60.07% |
Bayer & Stamm[ | 84.55% | 73.72% | 82.52% | 70.67% |
MesoNet[ | 87.27% | 56.20% | 61.17% | 40.67% |
XceptionNet[ | 95.15% | 83.48% | 92.09% | 77.89% |
本文方法 | 95.36% | 85.47% | 92.18% | 78.19% |
[1] | SUWAJANAKORN S , SEITZ S M , KEMELMACHER-SHLIZERMAN I , . Synthesizing Obama:learning lip sync from audio[J]. ACM Transactions on Graphics (TOG), 2017,36: 1-13. |
[2] | SIAROHIN A , LATHUILIèRE S , TULYAKOV S ,et al. First order motion model for image animation[J]. ArXiv, 2019,abs/2003.00196. |
[3] | YI R , YE Z , ZHANG J ,et al. Audio-driven talking face video generation with learning-based personalized head pose[J]. arXiv:2002.10137v2, 2020. |
[4] | PRAJWAL K R , MUKHOPADHYAY R , NAMBOODIRI V P ,et al. A lip sync expert is all you need for speech to lip generation in the wild[C]// Proceedings of the 28th ACM International Conference on Multimedia. 2020: 484-492. |
[5] | HALIASSOS A , VOUGIOUKAS K , PETRIDIS S ,et al. Lips don't lie:a generalisable and robust approach to face forgery detection[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021: 5037-5047. |
[6] | FARHA Y A , GALL J . MS-TCN:multi-stage temporal convolutional network for action segmentation[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 3570-3579. |
[7] | QIAN Y Y , YIN G J , SHENG L ,et al. Thinking in frequency:face forgery detection by mining frequency-aware clues[C]// Proceedings of Computer Vision – ECCV 2020. 2020: 86-103. |
[8] | LI J M , XIE H T , LI J H ,et al. Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021: 6454-6463. |
[9] | CHEN S , YAO T P , CHEN Y ,et al. Local relation learning for face forgery detection[J]. arXiv:2105.02577, 2021. |
[10] | REN S Q , HE K M , GIRSHICK R ,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149. |
[11] | HUANG G , LIU Z , VAN DER MAATEN L ,et al. Densely connected convolutional networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017: 2261-2269. |
[12] | SUN Y , WANG X G , TANG X O . Deep learning face representation from predicting 10,000 classes[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014: 1891-1898. |
[13] | WEN Y D , ZHANG K P , LI Z F ,et al. A discriminative feature learning approach for deep face recognition[C]// Proceedings of Computer Vision – ECCV 2016. 2016: 499-515. |
[14] | R?SSLER A , COZZOLINO D , VERDOLIVA L ,et al. FaceForensics++:learning to detect manipulated facial images[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019: 1-11. |
[15] | LI Y Z , YANG X , SUN P ,et al. Celeb-DF:a large-scale challenging dataset for DeepFake forensics[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 3204-3213. |
[16] | Faceswap. Faceswap github[EB]. |
[17] | THIES J , ZOLLH?FER M , STAMMINGER M ,et al. Face 2 face:real-time face capture and reenactment of RGB videos[J]. ArXiv, 2019,abs/2007.14808. |
[18] | DeepFakes. Deepfakes github[EB]. |
[19] | THIES J , ZOLLH?FER M , NIE?NER M . Real-time expression transfer for facial reenactment[J]. ACM Transactions on Graphics, 2015,34(6): 1-14. |
[20] | LI L Z , BAO J M , ZHANG T ,et al. Face X-ray for more general face forgery detection[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 5000-5009. |
[21] | KINGMA D P , BA J . Adam:a method for stochastic optimization[J]. CoRR. 2015,abs/1412.6980. |
[22] | FRIDRICH J J , KODOVSKY J , . Rich models for steganalysis of digital images[J]. IEEE Transactions on Information Forensics and Security, 2012,7(3): 868-882. |
[23] | AFCHAR D , NOZICK V , YAMAGISHI J ,et al. MesoNet:a compact facial video forgery detection network[J]. 2018 IEEE International Workshop on Information Forensics and Security (WIFS), 2018: 1-7. |
[24] | CHOLLET F , . Xception:deep learning with depthwise separable convolutions[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1800-1807. |
[25] | COZZOLINO D , POGGI G , VERDOLIVA L . Recasting residual-based local descriptors as convolutional neural networks:an application to image forgery detection[C]// Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security. 2017: 159-164. |
[26] | BAYAR B , STAMM M C . A deep learning approach to universal image manipulation detection using a new convolutional layer[C]// Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security. 2016: 5-10. |
[27] | RAHMOUNI N , NOZICK V , YAMAGISHI J ,et al. Distinguishing computer graphics from natural images using convolution neural networks[J]. 2017 IEEE Workshop on Information Forensics and Security (WIFS), 2017: 1-6. |
[28] | ZHOU P , HAN X T , MORARIU V I ,et al. Two-stream neural networks for tampered face detection[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017: 1831-1839. |
[29] | NGUYEN H H , FANG F M , YAMAGISHI J ,et al. Multi-task learning for detecting and segmenting manipulated facial images and videos[C]// Proceedings of 2019 IEEE 10th International Conference on Biometrics Theory,Applications and Systems. 2019: 1-8. |
[30] | MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose deepfakesand face manipulations[J]. 2019 IEEE Winter Applications of Computer Vision Workshops(WACVW), 2019: 83-92. |
[31] | LI Y , LYU S . Exposing DeepFake videos by detecting face warping artifacts[J]. ArXiv, 2019,abs/1811.00656. |
[1] | Xiaomeng LI, Daidou GUO, Xunfang ZHUO, Heng YAO, Chuan QIN. Carrier-independent screen-shooting resistant watermarking based on information overlay superimposition [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 135-149. |
[2] | Rongna XIE, Zhuhong MA, Zongyu LI, Ye TIAN. Encrypted traffic classification method based on convolutional neural network [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 84-91. |
[3] | Dengyong ZHANG, Huang WEN, Feng LI, Peng CAO, Lingyun XIANG, Gaobo YANG, Xiangling DING. Image inpainting forensics method based on dual branch network [J]. Chinese Journal of Network and Information Security, 2022, 8(6): 110-122. |
[4] | Jinyin CHEN, Changan WU, Haibin ZHENG. Novel defense based on softmax activation transformation [J]. Chinese Journal of Network and Information Security, 2022, 8(2): 48-63. |
[5] | Baolin QIU, Ping YI. Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation [J]. Chinese Journal of Network and Information Security, 2022, 8(2): 88-99. |
[6] | Lijuan LI, Man LI, Hongjun BI, Huachun ZHOU. Multi-type low-rate DDoS attack detection method based on hybrid deep learning [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 73-85. |
[7] | Zhongyuan QIN, Zhaoxiang HE, Tao LI, Liquan CHEN. Adversarial example defense algorithm for MNIST based on image reconstruction [J]. Chinese Journal of Network and Information Security, 2022, 8(1): 86-94. |
[8] | Deqing ZOU, Xiang LI, Minhuan HUANG, Xiang SONG, Hao LI, Weiming LI. Intelligent vulnerability detection system based on graph structured source code slice [J]. Chinese Journal of Network and Information Security, 2021, 7(5): 113-122. |
[9] | Zhenglong WANG, Baowen ZHANG. Survey of generative adversarial network [J]. Chinese Journal of Network and Information Security, 2021, 7(4): 68-85. |
[10] | Binglong LI, Jinlong TONG, Yu ZHANG, Yifeng SUN, Qingxian WANG, Chaowen CHANG. Auto forensic detecting algorithms of malicious code fragment based on TensorFlow [J]. Chinese Journal of Network and Information Security, 2021, 7(4): 154-163. |
[11] | Qingyin TAN, Yingming ZENG, Ye HAN, Yijing LIU, Zheli LIU. Survey on backdoor attacks targeted on neural network [J]. Chinese Journal of Network and Information Security, 2021, 7(3): 46-58. |
[12] | Luhui YANG,Huiwen BAI,Guangjie LIU,Yuewei DAI. Lightweight malicious domain name detection model based on separable convolution [J]. Chinese Journal of Network and Information Security, 2020, 6(6): 112-120. |
[13] | Ximeng LIU,Lehui XIE,Yaopeng WANG,Xuru LI. Adversarial attacks and defenses in deep learning [J]. Chinese Journal of Network and Information Security, 2020, 6(5): 36-53. |
[14] | Sijia DU,Haining YU,Hongli ZHANG. Survey of text classification methods based on deep learning [J]. Chinese Journal of Network and Information Security, 2020, 6(4): 1-13. |
[15] | Mingfang ZHAI,Xingming ZHANG,Bo ZHAO. Survey of encrypted malicious traffic detection based on deep learning [J]. Chinese Journal of Network and Information Security, 2020, 6(3): 66-77. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||
|