一种渐进式增长条件生成对抗网络模型

doi:10.11959/j.issn.1000-0801.2023134

摘要/Abstract

摘要：

渐进式增长生成对抗网络（PGGAN）是一种能够生成高分辨图像的网络模型，但是当样本间的类别不平衡或者样本类别过于相似或不相似时，容易出现模式崩溃现象而导致生成效果不佳。提出一种渐进式增长条件生成对抗网络（PGCGAN）模型，将条件生成对抗网络的思想引入PGGAN，在PGGAN的基础上加入类别信息作为条件，在网络结构和小批量标准差两个方面对 PGGAN 进行了改进，缓解图像生成过程中的模式崩溃现象。在对3个数据集的实验中，相比于PGGAN，PGCGAN在起始分数（IS）和Fréchet距离（FID）两个评价图像生成的指标方面都有较大程度的提升，生成的图像具有更高的多样性和真实性；且PGCGAN可以同时训练多个无关联的数据集而不崩溃，在类别不平衡或数据过于相似和不相似的数据集中均能产生高质量的图像。

关键词: 生成对抗网络, 渐进式增长条件生成对抗网络, 小批量标准差, 图像生成

Abstract:

Progressive growing of generative adversarial networks (PGGAN) is an adversarial network model that can generate high-resolution images.However, when the categories of samples are unbalanced, or the categories of samples are too similar or too dissimilar, it is prone to produce mode collapse, resulting in poor image generation effect.A progressive growing of conditional generative adversarial networks (PGCGAN) model was proposed.The idea of conditional generative adversarial networks (CGAN) was introduced into PGGAN.Using category information as condition, PGGAN was improved in two aspects of network structure and mini-batch standard deviation, and the phenomenon of model collapse in the process of image generation was alleviated.In the experiments on the three data sets, compared with PGGAN, PGCGAN has a greater degree of improvement in inception score and Fréchet inception distance, two evaluation indicators for image generation, and the generated images have higher diversity and authenticity; and PGCGAN multiple unrelated datasets can be trained simultaneously without crashing, and high-quality images can be produced in datasets with imbalanced categories or data that are too similar and dissimilar.

Key words: generative adversarial network, progressive growing of conditional GAN, mini-batch standard deviation, image generation

中图分类号:

TP391

马辉, 王瑞琴, 杨帅. 一种渐进式增长条件生成对抗网络模型[J]. 电信科学, 2023, 39(6): 105-113.

Hui MA, Ruiqin WANG, Shuai YANG. A progressive growing of conditional generative adversarial networks model[J]. Telecommunications Science, 2023, 39(6): 105-113.

图/表 8

图1

图2

图3

表1

表2

图4

图5

图6

参考文献 33

[1]	HARSHVARDHAN G M , GOURISARIA M K , PANDEY M ,et al. A comprehensive survey and analysis of generative models in machine learning[J]. Computer Science Review, 2020(38): 100285.
[2]	OUSSIDI A , ELHASSOUNY A . Deep generative models:survey[C]// Proceedings of 2018 International Conference on Intelligent Systems and Computer Vision (ISCV). Piscataway:IEEE Press, 2018: 1-8.
[3]	SHAHAM T R , DEKEL T , MICHAELI T . SinGAN:learning a generative model from a single natural image[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2019: 4569-4579.
[4]	于海涛, 杨小汕, 徐常胜 . 基于多模态输入的对抗式视频生成方法[J]. 计算机研究与发展, 2020,57(7): 1522-1530.
	YU H T , YANG X S , XU C S . Antagonistic video generation method based on multimodal input[J]. Journal of Computer Research and Development, 2020,57(7): 1522-1530.
[5]	ZHU J Y , PARK T , ISOLA P ,et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2017: 2242-2251.
[6]	YUAN W , NEUBIG G , LIU P . BARTScore:evaluating generated text as text generation[J]. Advances in Neural Information Processing Systems, 2021(34): 27263-27277.
[7]	LIAO J W , SHI Y , GONG M ,et al. Generating human readable transcript for automatic speech recognition with pre-trained language model[C]// Proceedings of ICASSP 2021 - 2021 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2021: 7578-7582.
[8]	陈华华, 陈哲 . 基于钉板分布稀疏变分自编码器的异常检测算法研究[J]. 电信科学, 2022,38(12): 65-77.
	CHEN H H , CHEN Z . Research on anomaly detection algorithm based on sparse variational autoencoder using spike and slab prior[J]. Telecommunications Science, 2022,38(12): 65-77.
[9]	JIAO P , GUO X , JING X ,et al. Temporal network embedding for link prediction via VAE joint attention mechanism[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021,33(12): 7400-7413.
[10]	HUANG D , ZHU X , LI R ,et al. Feature screening for network autoregression model[J]. Statistica Sinica, 2021(31): 1239.
[11]	AGGARWAL A , MITTAL M , BATTINENI G . Generative adversarial network:an overview of theory and applications[J]. International Journal of Information Management Data Insights, 2021,1(1): 100004.
[12]	宋珂慧, 张莹, 张江伟 ,等. 基于生成式对抗网络的结构化数据表生成模型[J]. 计算机研究与发展, 2019,56(9): 1832-1842.
	SONG K H , ZHANG Y , ZHANG J W ,et al. A generative model for synthesizing structured datasets based on GAN[J]. Journal of Computer Research and Development, 2019,56(9): 1832-1842.
[13]	MAO X D , LI Q , XIE H R ,et al. Least squares generative adversarial networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2017: 2813-2821.
[14]	ARJOVSKY M , BOTTOU L . Towards principled methods for training generative adversarial networks[J]. Stat, 2017:1050.
[15]	ADLER J , LUNZ S . Banach Wasserstein GAN[J]. Advances in Neural Information Processing Systems, 2018(31).
[16]	GULRAJANI I , AHMED F , ARJOVSKY M ,et al. Improved training of wasserstein GANs[J]. Advances in Neural Information Processing Systems, 2017(30).
[17]	WU J Q , HUANG Z W , THOMA J ,et al. Wasserstein divergence for GANs[C]// Proceedings of European Conference on Computer Vision. Cham:Springer, 2018: 673-688.
[18]	RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint, 2015,arXiv:1511.06434.
[19]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[20]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. Computer Science, 2014: 2672-2680.
[21]	CHEN X , DUAN Y , HOUTHOOFT R ,et al. InfoGAN:interpretable representation learning by information maximizing generative adversarial nets[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. New York:ACM Press, 2016: 2180-2188.
[22]	ODENA A , OLAH C , SHLENS J . Conditional image synthesis with auxiliary classifier GANs[C]// Proceedings of the 34th International Conference on Machine Learning - Volume 70. New York:ACM Press, 2017: 2642-2651.
[23]	LUCIC M , TSCHANNEN M , RITTER M ,et al. High-fidelity image generation with fewer labels[J]. arXiv preprint, 2019,arXiv:1903.02271.
[24]	LIU S , WANG T Z , BAU D ,et al. Diverse image generation via self-conditioned GANs[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE Press, 2020: 14286-14295.
[25]	ZHANG X , CHENG Z , ZHANG X ,et al. Posterior promoted GAN with distribution discriminator for unsupervised image synthesis[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 6519-6528.
[26]	KARRAS T , AILA , LAINE S ,et al. Progressive growing of GANs for improved quality,stability,and variation[J]. arXiv preprint, 2017,arXiv:1710.10196.
[27]	SALIMANS T , GOODFELLOW I , ZAREMBA W ,et al. Improved techniques for training GANs[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. New York:ACM Press, 2016: 2234-2242.
[28]	ODENA A , DUMOULIN V , OLAH C . Deconvolution and checkerboard artifacts[J]. Distill, 2016,1(10): e3.
[29]	GLOROT X , BENGIO Y . Understanding the difficulty of training deep feedforward neural networks[C]// Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.JMLR Workshop and Conference Proceedings. 2010: 249-256.
[30]	HUGHES D P , SALATHE M . An open access repository of images on plant health to enable the development of mobile disease diagnostics[J]. Computer Science, 2015,arXiv:1511.08060.
[31]	CHOI Y , UH Y , YOO J ,et al. StarGAN v2:diverse image synthesis for multiple domains[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 8188-8197.
[32]	CHONG M J , FORSYTH D . Effectively unbiased FID and inception score and where to find them[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 6069-6078.
[33]	HEUSEL M , RAMSAUER H , UNTERTHINER T ,et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York:ACM Press, 2017: 6629-6640.

模型	PlantVillage	AFHQ	Fingers
PGGAN	6.83	5.34	3.67
PGCGAN	8.50	10.09	7.64

模型	PlantVillage	AFHQ	Fingers
PGGAN	63.36	86.90	97.43
PGCGAN	33.72	36.62	42.34