基于深度学习的信源信道联合编码方法综述

doi:10.11959/j.issn.1000-0801.2020290

摘要/Abstract

摘要：

信息论的经典结果表明，信源信道分离编码是渐进最优的。但现代通信系统对时延、带宽等愈发敏感，分离设计对解码具有无限计算能力这一假设难以成立。带宽有限时，相对于信源信道联合编码，分离编码已被证明是次优的。传统的联合信源信道编码需要复杂的编码方案，相较之下，数据驱动的深度学习技术则带来了新的设计思路。适时地对相关研究成果进行总结，有助于进一步明确深度学习方法解决信源信道联合编码问题的方式，为研究新的研究方向提供依据。首先介绍了基于深度学习的信源压缩方案和端对端收发信机模型，随后分析不同信源类型下的两种联合编码设计思路，最后探讨了基于深度学习的信源信道联合编码的潜在问题和未来的工作方向。

关键词: 深度学习, 图像/视频压缩, 端到端收发信机, 信源信道联合编码

Abstract:

Classical information theory shows that separate source-channel coding is asymptotically optimal over a point-to-point channel.As modern communication systems are becoming more sensitive to delays and bandwidth,it becomes difficult to adopt the assumption that such separate designs have unlimited computing power for encoding and decoding.Compared to joint source-channel coding,separate coding has proven to be sub-optimal when the bandwidth is limited.However,conventional joint source-channel coding schemes require complicated design.In contrast,data-driven deep learning brings new designing ideas into the paradigm.A summary of relevant research results was provided,which will help to clarify the way in which deep learning methods solve the joint source-channel coding problem and to provide an overviewof new research directions.Source compression schemes and end-to-end communication system models were firstly introduced,both based on deep learning,then two kinds of joint coding designs under different types of source,and potential problems of joint source-channel coding based on deep learning and possible future research directions were introduced.

Key words: deep learning, image/video compression, end-to-end transceiver, joint source channel coding

中图分类号:

TN929.5

穆天杰,陈晓辉,汪逸云,马陆鹏,刘东,周晶,张文逸. 基于深度学习的信源信道联合编码方法综述[J]. 电信科学, 2020, 36(10): 56-66.

Tianjie MU,Xiaohui CHEN,Yiyun WANG,Lupeng MA,Dong LIU,Jing ZHOU,Wenyi ZHANG. A survey on deep learning based joint source-channel coding[J]. Telecommunications Science, 2020, 36(10): 56-66.

图/表 6

图1

图2

图3

图

图5

表1

参考文献 69

[11]	AOUDIA F A , HOYDIS J . End-to-end learning of communications systems without a channel model[C]// Proceedings of 2018 52nd Asilomar Conference on Signals,Systems,and Computers. Piscataway:IEEE Press, 2018: 298-303.
[12]	FELIX A , CAMMERER S , D?RNER S ,et al. OFDM-autoencoder for end-to-end learning of communications systems[C]// Proceedings of 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). Piscataway:IEEE Press, 2018: 1-5.
[13]	O'SHEA T J , ROY T , WEST N ,et al. Physical layer communications system design over-the-air using adversarial networks[C]// Proceedings of 2018 26th European Signal Processing Conference (EUSIPCO).[S.l.:s.n]. 2018: 529-532.
[14]	KANKANAHALLI S , . End-to-end optimized speech coding with deep neural networks[C]// Proceedings of 2018 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2018: 2521-2525.
[15]	AVINESH P , MEYER C M . Data-efficient neural text compression with interactive learning[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long and Short Papers).[S.l.:s.n]. 2019: 2543-2554.
[16]	BALLé J , MINNEN D , SINGH S ,et al. Variational image compression with a scale hyperprior[J]. arXiv:1802.01436, 2018
[17]	BALLé J , LAPARRA V , SIMONCELLI E P . End-to-end optimization of nonlinear transform codes for perceptual quality[C]// Proceedings of 2016 Picture Coding Symposium (PCS).[S.l.:s.n]. 2016: 1-5.
[18]	MINNEN D , BALLé J , TODERICI G D . Joint autoregressive and hierarchical priors for learned image compression[C]// Proceedings of Advances in Neural Information Processing Systems.[S.l.:s.n]. 2018: 10771-10780.
[1]	GAMAL A E , KIM Y H . Network information theory[M]. Cambridge: Cambridge University PressPress, 2011.
[2]	CUI H , LUO C , CHEN C W ,et al. Robust uncoded videotransmission over wireless fast fading channel[C]// Proceedings of IEEE INFOCOM 2014-IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2014: 73-81.
[3]	JAKUBCZAK S , KATABI D . SoftCast:clean-slate scalable wireless video[C]// Proceedings of the 2010 ACM Workshop on Wireless of the Students,by the Students,for the Students. New York:ACM Press, 2010: 9-12.
[4]	KATSAGGELOS A K , . Joint source-channel coding for video communications[C]// Handbook of Image and Video Processing. London:Elsevier,Academic Press, 2005: 1065-1082.
[5]	BOURTSOULATZE E , KURKA D B , GUNDUZ D . Deep joint source-channel coding for wireless image transmission[C]// ICASSP 2019-2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2019: 4774-4778.
[6]	KURKA D B ， GüNDüZ D . DeepJSCC-f:deep joint-source channel coding of images with feedback[J]. arXiv preprint arXiv:11174, 2019
[19]	TODERICI G , O'MALLEY S M , HWANG S J ,et al. Variable rate image compression with recurrent neural networks[J]. arXiv:1511.06085, 2015
[20]	TODERICI G , VINCENT D , JOHNSTON N ,et al. Full resolution image compression with recurrent neural networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 5306-5314.
[7]	BALLé J , LAPARRA V , SIMONCELLI E P . End-to-end optimized image compression[J]. arXiv preprint arXiv:1611.01704, 2016
[8]	JOHNSTON N , VINCENT D , MINNEN D ,et al. Improved lossy image compression with priming and spatially adaptive bit rates for recurrent networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018.
[21]	AGUSTSSON E , TSCHANNEN M , MENTZERF ,et al. Generative adversarial networks for extreme learned image compression[C]// Proceedings of the IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2019.
[22]	LIU D , LI Y , LIN J ,et al. Deep learning-based video coding:a review and a case study[J]. ACM Computing Surveys (CSUR), 2020(53): 1-35.
[9]	LU G , OUYANG W , XU D ,et al. Dvc:an end-to-end deep video compression framework[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 11006-11015.
[10]	RIPPEL O , NAIR S , LEW C ,et al. Learned video compression[C]// Proceedings of the IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 3454-3463.
[23]	LIU D , LI Y , LIN J ,et al. Deep learning-based video coding:a review and a case study[J]. arXiv:Multimedia, 2019
[24]	LIN J , LIU D , LI H ,et al. Generative adversarial network-based frame extrapolation for video coding[C]// Proceedings of 2018 IEEE Visual Communications and Image Processing (VCIP). Piscataway:IEEE Press, 2018: 1-4.
[25]	O’SHEA T , HOYDIS J . An introduction to deep learning for the physical layer[J]. IEEE Transactions on Cognitive Communications and Networking, 2017(3): 563-575.
[26]	GüNDüZ D , KERRET P D , SIDIROPOULOS N D ,et al. Machine learning in the air[J]. IEEE Journal on Selected Areas in Communications, 2019(37): 2184-2199.
[27]	GOLDSMITH A , . Joint source/channel coding for wireless channels[C]// Proceedings of 1995 IEEE 45th Vehicular Technology Conference.Countdown to the Wireless Twenty-First Century. Piscataway:IEEE Press, 1995: 614-618.
[28]	ZEHAVI E . 8-PSK trellis codes for a Rayleigh channel[J]. IEEE Transactions on Communications, 1992(40): 873-884.
[29]	WYMEERSCH H . Iterative receiver Design[M]. Cambridge: Cambridge University PressPress, 2007.
[30]	O’SHEA T , HOYDIS J . An introduction to deep learning for the physical layer[J]. IEEE Transactions on Cognitive Communications, 2017(3): 563-575.
[31]	张静, 金石, 温朝凯 ,等. 基于人工智能的无线传输技术最新研究进展[J]. 电信科学, 2018,34(8): 46-55.
	ZHANG J , JIN S , WEN C K ,et al. An overview of wireless transmission technology utilizing artificial intelligence[J]. Telecommunications Science, 2018,34(8): 46-55.
[32]	O’SHEA T J , ERPEK T , CLANCY T C . Deep learning based MIMO communications[J]. arXiv preprint arXiv:07980, 2017
[33]	ERPEK T , O'SHEA T J , CLANCY T C . Learning a physical layer scheme for the MIMO interference channel[C]// Proceedings of 2018 IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2018: 1-5.
[34]	YE H , LIANG L , LI G Y ,et al. Deep learning based end-to-end wireless communication systems with conditional GAN as unknown channel[J]. arXiv:02551, 2019
[35]	FINN C , ABBEEL P , LEVINE S ,et al. Model-agnostic meta-learning for fast adaptation of deep networks[J]. 2017
[36]	CAIRE G , TARICCO G , BIGLIERI E J I . Bit-interleaved coded modulation[J]. IEEE Transactions on Information Theory, 1998(44): 927-946.
[37]	GOBLICKTJ . Theoretical limitations on the transmission of data from analog sources[M]. New York: ACM PressPress, 1965: 558-567.
[38]	TUNG T Y GüNDüZ D . SparseCast:hybrid digital-analog wireless image transmission exploiting frequency-domain sparsity[J]. IEEE Communications Letters, 2018(22): 2451-2454.
[39]	NEAL R M J A I . Connectionist learning of belief networks[J]. Artificial Intelligence, 1992,56(1): 71-113.
[40]	ROLFE J TJ A P A . Discrete variational autoencoders[J]. arXiv:1609.02200, 2016
[41]	OORD A V D , VINYALS O . Neural discrete representation learning[C]// Proceedings of Advances in Neural Information Processing Systems.[S.l.:s.n]. 2017: 6306-6315.
[42]	CARPI F , H?GER F C , MARTALò M . Reinforcement learning for channel coding:learned bit-flipping decoding[C]// Proceedings of 2019 57th Annual Allerton Conference on Communication,Control,and Computing (Allerton).[S.l.:s.n]. 2019: 922-929.
[43]	JIANG Y , KANNAN S , KIM H ,et al. Deepturbo:deep turbo decoder[C]// Proceedings of 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). Piscataway:IEEE Press, 2019: 1-5.
[44]	KURKA D B , GüNDüZ D , . Successive refinement of images with deep joint source-channel coding[C]// Proceedings of 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). Piscataway:IEEE Press, 2019: 1-5.
[45]	JANKOWSKI M , GüNDüZ D , MIKOLAJCZYK K . Deep joint source-channel coding for wireless image retrieval[C]// Proceedings of ICASSP 2020-2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2020: 5070-5074.
[46]	ZARCONE R , PAITON D , ANDERSON A ,et al. Joint source-channel coding with neural networks for analog data compression and storage[C]// Proceedings of 2018 Data Compression Conference.[S.l.:s.n]. 2018: 147-156.
[47]	CHOI K , TATWAWADI K , WEISSMAN T ,et al. NECST:neural joint source-channel coding[Z]. 2018.
[48]	SONG Y , XU M , YU l ,et al. Infomax neural joint source-channel coding via adversarial bit flip[Z]. 2020.
[49]	JIANG F , TAO W , LIU S ,et al. An end-to-end compression framework based on convolutional neural networks[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017(28): 3007-3018.
[50]	SHAO J , ZHANG J . Bottlenet++:an end-to-end approach for feature compression in device-edge co-inference systems[C]// Proceedings of 2020 IEEE International Conference on Communications Workshops (ICC Workshops). Piscataway:IEEE Press, 2020: 1-6.
[51]	FARSAD N , RAO M , GOLDSMITH A . Deep learning for joint source-channel coding of text[C]// Proceedings of 2018 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2018: 2326-2330.
[52]	XUAN Z , NARAYANAN K . Analog joint source-channel coding for gaussian sources over AWGN channels with deep learning[C]// Proceedings of 2020 International Conference on Signal Processing and Communications (SPCOM). Piscataway:IEEE Press, 2020: 1-5.
[53]	KURKA D B,GüNDüZ D . DeepJSCC-f:deep joint source-channel coding of images with feedback[Z]. 2020.
[54]	KURKA D B,GüNDüZ D . Bandwidth-agile image transmission with deep joint source-channel coding[Z]. 2020.
[55]	SAIDUTTA Y M , ABDI A , FEKRI F . M to 1 joint source-channel coding of gaussian sources via dichotomy of the input space based on deep learning[C]// Proceedings of 2019 Data Compression Conference (DCC). Piscataway:IEEE Press, 2019: 488-497.
[56]	SAIDUTTA Y M , ABDI S , FEKRI F . Joint source-channel coding for gaussian sources over AWGN channels using variational autoencoders[C]// Proceedings of 2019 IEEE International Symposium on Information Theory (ISIT). Piscataway:IEEE Press, 2019: 1327-1331.
[57]	SAIDUTTA T M , ABDI A , FEKRI F . Joint source-channel coding of gaussian sources over AWGN channels via manifold variational autoencoders[C]// Proceedings of 2019 57th Annual Allerton Conference on Communication,Control,and Computing (Allerton).[S.l.:s.n]. 2019: 514-520.
[58]	DUAN Y , LU J , WANG Z ,et al. Learning deep binary descriptor with multi-quantization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019(41): 1924-1938.
[59]	ZHANG Z , ZHAO Y , LIU J ,et al. A general deep learning framework for network reconstruction and dynamics learning[J]. Applied Network Science, 2019(4): 1-17.
[60]	MENA G , BELANGER D , LINDERMAN S ,et al. Learning latent permutations with gumbel-sinkhorn networks[J]. arXiv:1802.08665, 2018
[61]	VAHDAT A , ANDRIYASH E , MACREADY W . Dvae#:discrete variational autoencoders with relaxed boltzmann priors[C]// Advances in Neural Information Processing Systems.[S.l.:s.n]. 2018: 1864-1874.
[62]	WILLIAMS R J J M L . Simple statistical gradient-following algorithms for connectionist reinforcement learning[J]. Machine Learning, 1992,8(3-4): 229-256.
[63]	WANG C , CHEN X , SMOLA A J ,et al. Variance reduction for stochastic gradient optimization[C]// Proceedings of Advances in Neural Information Processing Systems.[S.l.:s.n]. 2013: 181-189.
[64]	GU S , LEVINE S , SUTSKEVER I ,et al. Muprop:unbiased backpropagation for stochastic neural networks[Z]. 2015.
[65]	RUIZ F R , AUEB M T R , BLEI D . The generalized reparameterization gradient[C]// Proceedings of Advances in Neural Information Processing systems.[S.l.:s.n]. 2016: 460-468.
[66]	TUCKER G , MNIH A , MADDISON C J ,et al. Rebar:low-variance,unbiased gradient estimates for discrete latent variable models[C]// Proceedings of Advances in Neural Information Processing Systems.[S.l.:s.n]. 2017: 2627-2636.
[67]	GRATHWOHL W , CHOI D , WU Y ,et al. Backpropagation through the void:optimizing control variates for black-box gradient estimation[Z]. 2017.
[68]	JANG E , GU S , POOLE B J . Categorical reparameterization with gumbel-softmax[Z]. 2016.
[69]	MADDISON C J , MNIH A , THE Y W J . The concrete distribution:a continuous relaxation of discrete random variables[Z]. 2016.

Deep JSCC	物理信道（AWGN，瑞利信道）	抽象信道（BSC，BEC）
结构化信源（图像，文本）	Gunduz团队^[5,6,38,44]，Mikolaj^[45]，Ryan^[46]	Kristy^[47]，Song^[48]，Shao^[48,49,50]，Nariman^[51]
非结构化信源（高斯信源）	?ashas^[51]，Krishna^[52]	Carpi^[42]，Jiang^[43]