面向语义通信的非线性变换编码

doi:10.11959/j.issn.1000-436x.2023087

Abstract

Abstract:

The modular design and limited processing mechanism of traditional communication systems limit the continuous improvement of end-to-end data transmission capability.For this reason, a new nonlinear transform coding framework for semantic communications was proposed.First, an end-to-end rate distortion optimization criterion for semantic communication was derived based on variational theory.Based on this, a nonlinear transform was designed to extract the compact representation of source data in the semantic latent space, and variable-rate nonlinear joint source-channel coding was implemented through the guidance of variational entropy model.Experiments show that semantic nonlinear transform coding can significantly improve the end-to-end data transmission performance and robustness, and is one of the key technologies to catalyze future semantic communications.

Key words: semantic communications, nonlinear transform, nonlinear coding, variational entropy modeling, rate-distortion optimization

CLC Number:

TN92

Ping ZHANG, Jincheng DAI, Yuming ZHANG, Sixian WANG, Xiaoqi QIN, Kai NIU. Nonlinear transform coding for semantic communications[J]. Journal on Communications, 2023, 44(4): 1-14.

Figures/Tables 12

References 35

[1]	WARREN W . Recent contributions to the mathematical theory of communication[J]. ETC:A Review of General Semantics, 2017,74(1/2): 136-157.
[2]	SHANNON C E . A mathematical theory of communication[J]. The Bell System Technical Journal, 1948,27(3): 379-423.
[3]	ZHANG P , XU W J , GAO H ,et al. Toward wisdom-evolutionary and primitive-concise 6G:a new paradigm of semantic communication networks[J]. Engineering, 2022,8: 60-73.
[4]	QIN Z J , TAO X M , LU J H ,et al. Semantic communications:principles and challenges[J]. arXiv Preprint,arXiv:2201.01389, 2022.
[5]	石光明, 李莹玉, 谢雪梅 . 语义通讯:智能时代的产物[J]. 模式识别与人工智能, 2018,31(1): 91-99.
	SHI G M , LI Y Y , XIE X M . Semantic communications:outcome of the intelligence era[J]. Pattern Recognition and Artificial Intelligence, 2018,31(1): 91-99.
[6]	石光明, 肖泳, 李莹玉 ,等. 面向万物智联的语义通信网络[J]. 物联网学报, 2021,5(2): 26-36.
	SHI G M , XIAO Y , LI Y Y ,et al. Semantic communication networking for the intelligence of everything[J]. Chinese Journal on Internet of Things, 2021,5(2): 26-36.
[7]	NIU K , DAI J C , YAO S S ,et al. A paradigm shift toward semantic communications[J]. IEEE Communications Magazine, 2022,60(11): 113-119.
[8]	牛凯, 戴金晟, 张平 ,等. 面向6G的语义通信[J]. 移动通信, 2021,45(4): 85-90.
	NIU K , DAI J C , ZHANG P ,et al. 6G-oriented semantic communications[J]. Mobile Communications, 2021,45(4): 85-90.
[9]	DAI J C , ZHANG P , NIU K ,et al. Communication beyond transmitting bits:semantics-guided source and channel coding[J]. IEEE Wireless Communications, 2022:doi.org/10.1109/MWC.017.2100705.
[10]	XIE H Q , QIN Z J , TAO X M ,et al. Task-oriented multi-user semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022,40(9): 2584-2597.
[11]	BALLé J , CHOU P A , MINNEN D ,et al. Nonlinear transform coding[J]. IEEE Journal of Selected Topics in Signal Processing, 2020,15(2): 339-353.
[12]	RICHARDSON T , KUDEKAR S . Design of low-density parity check codes for 5G new radio[J]. IEEE Communications Magazine, 2018,56(3): 28-34.
[13]	ARIKAN E . Channel polarization:a method for constructing capacity-achieving codes for symmetric binary-input memoryless channels[J]. IEEE Transactions on Information Theory, 2009,55(7): 3051-3073.
[14]	FARSAD N , RAO M , GOLDSMITH A . Deep learning for joint source-channel coding of text[C]// Proceedings of 2018 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2018: 2326-2330.
[15]	CHOI K , TATWAWADI K , GROVER A ,et al. Neural joint source-channel coding[C]// International Conference on Machine Learning. New York:PMLR, 2019: 1182-1192.
[16]	BOURTSOULATZE E , KURKA D B , GüNDüZ D , . Deep joint source-channel coding for wireless image transmission[C]// Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2019: 4774-4778.
[17]	GüNDüZ D , KURKA D B , JANKOWSKI M ,et al. Communicate to learn at the edge[J]. IEEE Communications Magazine, 2020,58(12): 14-19.
[18]	KURKA D B , GüNDüZ D , . Bandwidth-agile image transmission with deep joint source-channel coding[J]. IEEE Transactions on Wireless Communications, 2021,20(12): 8081-8095.
[19]	XU J L , AI B , CHEN W ,et al. Wireless image transmission using deep source channel coding with attention modules[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022,32(4): 2315-2328.
[20]	WANG S X , YANG K , DAI J C ,et al. Distributed image transmission using deep joint source-channel coding[C]// Proceedings of 2022 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2022: 5208-5212.
[21]	WANG J , WANG S X , DAI J C ,et al. Perceptual learned source-channel coding for high-fidelity image semantic transmission[J]. arXiv Preprint,arXiv:2205.13120, 2022.
[22]	DAI J C , WANG S X , TAN K L ,et al. Nonlinear transform source-channel coding for semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022,40(8): 2300-2316.
[23]	WANG Z , SIMONCELLI E P , BOVIK A C . Multiscale structural similarity for image quality assessment[C]// Proceedings of the Thirty Seventh Asilomar Conference on Signals,Systems ＆ Computers. Piscataway:IEEE Press, 2004: 1398-1402.
[24]	ZHANG R , ISOLA P , EFROS A A ,et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 586-595.
[25]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. arXiv Preprint,arXiv:1411.1784. 2014.
[26]	DOSOVITSKIY A , BEYER L , KOLESNIKOV A ,et al. An image is worth 16×16 words:transformers for image recognition at scale[J]. arXiv Preprint,arXiv:2010.11929. 2020.
[27]	LIU Z , LIN Y T , CAO Y ,et al. Swin transformer:hierarchical vision transformer using shifted windows[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2022: 9992-10002.
[28]	BA J L , KIROS J R , HINTON G E . Layer normalization[J]. arXiv Preprint,arXiv:1607.06450. 2016.
[29]	HAN Y Z , HUANG G , SONG S J ,et al. Dynamic neural networks:a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022,44(11): 7436-7456.
[30]	XIAO Z X , YAO S S , DAI J C ,et al. Wireless deep speech semantic transmission[J]. arXiv Preprint,arXiv:2211.02283. 2022.
[31]	WANG S X , DAI J C , LIANG Z J ,et al. Wireless deep video semantic transmission[J]. IEEE Journal on Selected Areas in Communications, 2023,41(1): 214-229.
[32]	KUZNETSOVA A , ROM H , ALLDRIN N ,et al. The open images dataset v4[J]. International Journal of Computer Vision, 2020,128(7): 1956-1981.
[33]	WALLACE G K . The JPEG still picture compression standard[J]. IEEE Transactions on Consumer Electronics, 1992,38(1): 30-44.
[34]	MARCELLIN M W , GORMISH M J , BILGIN A ,et al. An overview of JPEG2000[C]// Proceedings of Data Compression Conference. Piscataway:IEEE Press, 2002: 523-541.
[35]	LIU L F , OESTGES C , POUTANEN J ,et al. The COST 2100 MIMO channel model[J]. IEEE Wireless Communications, 2012,19(6): 92-99.

Metrics

Recommended 0

No Suggested Reading articles found!

Nonlinear transform coding for semantic communications

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 35

Related Articles 5

Metrics

Recommended 0

[1]	Ping ZHANG, Kai NIU, Shengshi YAO, Jincheng DAI. Semantic communications for future: basic principle and implementation methodology [J]. Journal on Communications, 2023, 44(5): 1-14.
[2]	Hongwei GUO, Ce ZHU, Xu YANG, Lei LUO. Temporal dependent rate-distortion optimization based on distortion backward propagation [J]. Journal on Communications, 2022, 43(12): 222-232.
[3]	Wei LI,Fan ZHAO,Er-hu ZHANG,Peng REN. Lagrange multiplier selection based on SATD for H.265/HEVC [J]. Journal on Communications, 2017, 38(6): 68-74.
[4]	Qian-nan LI,Yun-qiang LI,Shu-jing JIANG,Yao LU. Research on differential properties of Keccak-like nonlinear transform [J]. Journal on Communications, 2012, 33(9): 140-146.
[5]	Yong-mei SUN,Tian-shuang QIU. SCOT weighted time delay estimation method under impulsive noise environment [J]. Journal on Communications, 2005, 26(12): 13-18.