融合小波快捷连接生成对抗网络的面部性别伪造

doi:10.11959/j.issn.2096-109x.2023046

Abstract

Abstract:

The mainstream methods in the field of facial attribute manipulation had the following two defects due to data and model architecture limitations.First, the bottleneck structure of the autoencoder model results in the loss of feature information, and the traditional method of continuously injected styles to the source domain features during the decoding process makes the generated image too referential to the target domain while losing the identity information and fine-grained details.Second, differences in facial attributes composition between images, such as gender, ethnicity, or age can cause variations in frequency domain information.And the current unsupervised training methods do not automatically adjust the proportion of source and target domain information in the style injection stage, resulting in artifacts in generated images.A facial gender forgery model based on generative adversarial networks and image-to-image translation techniques, namely fused wavelet shortcut connection generative adversarial network (WscGAN), was proposed to address the these issues.Shortcut connections were added to the autoencoder structure, and the outputs of different encoding stages were decomposed at the feature level by wavelet transform.Attention mechanism was employed to process them one by one, to dynamically change the proportion of source domain features at different frequencies in the decoding process.This model could complete forgery of facial images in terms of gender attributes.To verify the effectiveness of the model, it was conducted on the CelebA-HQ dataset and the FFHQ dataset.Compared with the existing optimal models, the method improves the FID and LPIPS indices by 5.4% and 11.2%, and by 1.8% and 6.7%, respectively.Furthermore, the effectiveness of the proposed method in improving the gender attribute conversion of facial images is fully demonstrated by the results based on qualitative visual comparisons.

Key words: image generation, generative adversarial network, image-to-image translation, facial attribute manipulation, wavelet transform

CLC Number:

TP37
TP391

Wanze CHEN, Liqing HUANG, Jiazhen CHEN, Feng YE, Tianqiang HUANG, Haifeng LUO. Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 150-160.

Figures/Tables 8

模型	CelebA-HQ		FFHQ
模型	FID	LPIPS	FID	LPIPS
MUNIT^[26]	24.96	0.252	28.03	0.231
DRIT++^[27]	52.13	0.178	44.12	0.061
StarGAN-v2^[39]	13.71	0.452	19.21	0.455
PHStarGAN-v2 ^[29]	16.63	0.332	43.03	0.342
本文模型	$12 . 26$	$0 . 462$	$14 . 95$	$0 . 480$

模型	CelebA-HQ		FFHQ
模型	FID	LPIPS	FID	LPIPS
MUNIT^[26]	64.54	0.123	51.29	0.126
DRIT++^[27]	53.3	0.311	40.08	0.032
StarGAN-v2^[39]	23.84	0.388	28.94	0.409
PHStarGAN-v2 ^[29]	28.11	0.290	56.82	0.332
本文模型	$22 . 06$	$0 . 393$	$23 . 90$	$0 . 420$

不同方法	潜在引导生成		参考引导生成
不同方法	FID	LPIPS	FID	LPIPS
自编码器	22.64	0.092	35.81	0.238
自编码器+快捷连接	19.98	0.125	$19 . 63$	0.124
自编码器+快捷连接+wsc模块	$13 . 26$	$0 . 462$	22.06	$0 . 393$

References 41

[1]	GOODFELLOW I , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial nets[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014: 2672-2680.
[2]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. arXiv Preprint arXiv:1411.1784, 2014.
[3]	TANG H , XU D , YAN Y ,et al. Local class-specific and global image-level generative adversarial networks for semantic-guided scene generation[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2020: 7867-7876.
[4]	CHEN B J , TAN W J , WANG Y T ,et al. Distinguishing between natural and GAN-generated face images by combining global and local features[J]. Chinese Journal of Electronics, 2022,31(1): 59-67.
[5]	GUO X , WANG Z , YANG Q ,et al. Gan-based virtual-to-real image translation for urban scene semantic segmentation[J]. Neurocomputing, 2020,394: 127-135.
[6]	KIM T , CHA M , KIM H ,et al. Learning to discover cross-domain relations with generative adversarial networks[J]. arXiv Preprint arXiv:1703.05192, 2017.
[7]	王旭东, 卫红权, 高超 ,等. 身份保持约束下的人脸图像补全[J]. 网络与信息安全学报, 2018,4(8): 71-76.
	WANG X D , WEI H Q , GAO C ,et al. Identity preserving face completion with generative adversarial networks[J]. Chinese Journal of Network and Information Security, 2018,4(8): 71-76.
[8]	TUNG H Y F , HARLEY A W , SETO W ,et al. Adversarial inverse graphics networks:learning 2D-to-3D lifting and image-to-image translation from unpaired supervision[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). 2017: 4364-4372.
[9]	LI X Y , ZHANG S C , HU J ,et al. Image-to-image translation via hierarchical style disentanglement[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021: 8635-8644.
[10]	ZHU J Y , PARK T , ISOLA P ,et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). 2017: 2242-2251.
[11]	TOMEI M , CORNIA M , BARALDI L ,et al. Art2Real:unfolding the reality of artworks via semantically-aware image-to-image translation[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 5842-5852.
[12]	ISOLA P , ZHU J Y , ZHOU T H ,et al. Image-to-image translation with conditional adversarial networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 5967-5976.
[13]	KINGMA D P , WELLING M . Auto-encoding variational bayes[J]. arXiv Preprint arXiv:1312.6114, 2013.
[14]	MAKHZANI A , SHLENS J , JAITLY N ,et al. Adversarial autoencoders"[J]. arXiv Preprint arXiv:1511.05644, 2015.
[15]	KARRAS T , AILA , LAINE S ,et al. Progressive growing of GANs for improved quality,stability,and variation[J]. arXiv preprint arXiv:1710.10196, 2017.
[16]	KARRAS T , LAINE S , AILA T M . A style-based generator architecture for generative adversarial networks[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 4396-4405.
[17]	DENTON E L , CHINTALA S , FERGUS R . Deep generative image models using a laplacian pyramid of adversarial networks[J]. arXiv preprint arXiv:1506.05751, 2015.
[18]	MAO X D , LI Q , XIE H R ,et al. Least Squares generative adversarial networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). 2017: 2813-2821.
[19]	ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein GAN[J]. arXiv Preprint arXiv:1701.07875, 2017.
[20]	MIYATO T , KATAOKA T , KOYAMA M ,et al. Spectral normalization for generative adversarial networks[J]. arXiv Preprint arXiv:1802.05957, 2018.
[21]	RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv Preprint arXiv:1511.06434, 2015.
[22]	CHEN X , DUAN Y , HOUTHOOFT R ,et al. InfoGAN:interpretable representation learning by information maximizing generative adversarial nets[C]// Proceedings of Neural Information Processing Systems. 2016: 2172-2180.
[23]	BROCK A , DONAHUE J , SIMONYAN K . Large scale GAN training for high fidelity natural image synthesis[J]. arXiv Preprint arXiv:1809.11096, 2018.
[24]	TANG H , XU D , SEBE N ,et al. Multi-channel attention selection GAN with cascaded semantic guidance for cross-view image translation[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 2412-2421.
[25]	PARK T , LIU M Y , WANG T C ,et al. Semantic image synthesis with spatially-adaptive normalization[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). 2020: 2332-2341.
[26]	HUANG X , LIU M Y , BELONGIE S ,et al. Multimodal unsupervised image-to-image translation[C]// Proceedings of European Conference on Computer Vision. 2018: 179-196.
[27]	LEE H Y , TSENG H Y , MAO Q ,et al. DRIT++:diverse image-to-image translation via disentangled representations[J]. arXiv Preprint arXiv:1905.01270, 2019.
[28]	CHOI Y , CHOI M , KIM M ,et al. StarGAN:unified generative adversarial networks for multi-domain image-to-image translation[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018: 8789-8797.
[29]	GRASSUCCI E , SIGILLO L , UNCINI A ,et al. Hyper complex image-to-image translation[J]. arXiv Preprint arXiv:2205.02087, 2022.
[30]	DEMIREL H , ANBARJAFARI G . Discrete wavelet transform-based satellite image resolution enhancement[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011,49(6): 1997-2004.
[31]	CHEN T S , LIN L , ZUO W M ,et al. Learning a wavelet-like auto-encoder to accelerate deep neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018,32(1).
[32]	HUANG H B , HE R , SUN Z N ,et al. Wavelet-SRNet:a wavelet-based CNN for multi-scale face super resolution[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). 2017: 1698-1706.
[33]	FUJIEDA S , TAKAYAMA K , HACHISUKA T . Wavelet convolutional neural networks for texture classification"[J]. arXiv Preprint arXiv:1707.07394, 2017.
[34]	WILLIAMS T , LI R . Wavelet pooling for convolutional neural networks[C]// Proceedings of International Conference on Learning Representations. 2018.
[35]	RONNEBERGER O , FISCHER P , BROX T . U-Net:convolutional networks for biomedical image segmentation[C]// Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015: 234-241.
[36]	HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). 2017: 1510-1519.
[37]	YOO J , UH Y , CHUN S ,et al. Photorealistic style transfer via wavelet transforms[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2020: 9035-9044.
[38]	MAO Q , LEE H Y , TSENG H Y ,et al. Mode seeking generative adversarial networks for diverse image synthesis[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 1429-1437.
[39]	CHOI Y , UH Y , YOO J ,et al. StarGAN v2:diverse image synthesis for multiple domains[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020: 8185-8194.
[40]	HEUSEL M , RAMSAUER H , UNTERTHINER T ,et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium[C]// Proceedings of Neural Information Processing Systems (NIPS). 2017: 6626-6637.
[41]	ZHANG R , ISOLA P , EFROS A A ,et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018: 586-595.

Metrics

Recommended 0

No Suggested Reading articles found!

Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 41

Related Articles 6

Metrics

Recommended 0

[1]	Feng YU, Qingxin LIN, Hui LIN, Xiaoding WANG. Privacy-enhanced federated learning scheme based on generative adversarial networks [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 113-122.
[2]	Yuguang YANG, Guodong CAO. New image encryption algorithm based on double chaotic system and compressive sensing [J]. Chinese Journal of Network and Information Security, 2022, 8(5): 88-97.
[3]	Zhenglong WANG, Baowen ZHANG. Survey of generative adversarial network [J]. Chinese Journal of Network and Information Security, 2021, 7(4): 68-85.
[4]	Yu ZHANG,Xixiang LYU,Yucong ZOU,Yige LI. Differentially private sequence generative adversarial networks for data privacy masking [J]. Chinese Journal of Network and Information Security, 2020, 6(4): 109-119.
[5]	Qiao-ling LI,Qing-xiao GUAN,Xian-feng ZHAO. Image generation classification method based on convolution neural network [J]. Chinese Journal of Network and Information Security, 2016, 2(9): 40-48.
[6]	Bo WANG,Jian-feng YIN,Ya-bin LI. Performance evaluation for JPEG compression on model-based source camera identification [J]. Chinese Journal of Network and Information Security, 2016, 2(9): 65-71.