面向语义通信的非线性变换编码

doi:10.11959/j.issn.1000-436x.2023087

通信学报 ›› 2023, Vol. 44 ›› Issue (4): 1-14.doi: 10.11959/j.issn.1000-436x.2023087

• 学术论文 • 下一篇

面向语义通信的非线性变换编码

张平¹, 戴金晟², 张育铭², 王思贤², 秦晓琦¹, 牛凯¹

¹ 北京邮电大学网络与交换技术国家重点实验室，北京 100876
² 北京邮电大学泛网无线通信教育部重点实验室，北京 100876

修回日期:2023-04-11 出版日期:2023-04-01 发布日期:2023-04-01
作者简介:张平（1959- ），男，陕西汉中人，博士，中国工程院院士，北京邮电大学教授、博士生导师，主要研究方向为先进移动通信系统
戴金晟（1992- ），男，湖南常德人，博士，北京邮电大学副教授、博士生导师，主要研究方向为语义通信、信源信道编码、智能信号处理等
张育铭（1999- ），男，河南南阳人，北京邮电大学硕士生，主要研究方向为语义通信等
王思贤（1997- ），男，安徽宿州人，北京邮电大学博士生，主要研究方向为语义通信、信源信道编码、计算机视觉等
秦晓琦（1988- ），女，北京人，博士，北京邮电大学副教授，主要研究方向为下一代无线网络、智能高效机器型通信等
牛凯（1976- ），男，山西文水人，博士，北京邮电大学教授，主要研究方向为信息论与极化码、智能信号处理等
基金资助:
国家自然科学基金资助项目(62293481);国家自然科学基金资助项目(92067202);国家自然科学基金资助项目(62001049);国家自然科学基金资助项目(62071058)

Nonlinear transform coding for semantic communications

Ping ZHANG¹, Jincheng DAI², Yuming ZHANG², Sixian WANG², Xiaoqi QIN¹, Kai NIU¹

¹ The State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
² The Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China

Revised:2023-04-11 Online:2023-04-01 Published:2023-04-01
Supported by:
The National Natural Science Foundation of China(62293481);The National Natural Science Foundation of China(92067202);The National Natural Science Foundation of China(62001049);The National Natural Science Foundation of China(62071058)

摘要/Abstract

摘要：

针对经典通信系统模块设计分离、处理范式受限等因素限制了端到端信息传输性能持续提升的问题，提出了面向语义通信的非线性变换编码传输新框架。首先，基于变分理论推导了语义通信端到端率失真优化准则。据此，设计了非线性变换来提取信源数据在语义隐空间的紧致表征，并通过语义变分熵建模引导实现了变速率非线性联合信源信道编码。实验表明，语义非线性变换编码能显著提升端到端数据传输性能及鲁棒性，是实现语义通信的关键技术之一。

关键词: 语义通信, 非线性变换, 非线性编码, 变分熵建模, 率失真优化

Abstract:

The modular design and limited processing mechanism of traditional communication systems limit the continuous improvement of end-to-end data transmission capability.For this reason, a new nonlinear transform coding framework for semantic communications was proposed.First, an end-to-end rate distortion optimization criterion for semantic communication was derived based on variational theory.Based on this, a nonlinear transform was designed to extract the compact representation of source data in the semantic latent space, and variable-rate nonlinear joint source-channel coding was implemented through the guidance of variational entropy model.Experiments show that semantic nonlinear transform coding can significantly improve the end-to-end data transmission performance and robustness, and is one of the key technologies to catalyze future semantic communications.

Key words: semantic communications, nonlinear transform, nonlinear coding, variational entropy modeling, rate-distortion optimization

中图分类号:

TN92

张平, 戴金晟, 张育铭, 王思贤, 秦晓琦, 牛凯. 面向语义通信的非线性变换编码[J]. 通信学报, 2023, 44(4): 1-14.

Ping ZHANG, Jincheng DAI, Yuming ZHANG, Sixian WANG, Xiaoqi QIN, Kai NIU. Nonlinear transform coding for semantic communications[J]. Journal on Communications, 2023, 44(4): 1-14.

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

参考文献 35

[1]	WARREN W . Recent contributions to the mathematical theory of communication[J]. ETC:A Review of General Semantics, 2017,74(1/2): 136-157.
[2]	SHANNON C E . A mathematical theory of communication[J]. The Bell System Technical Journal, 1948,27(3): 379-423.
[3]	ZHANG P , XU W J , GAO H ,et al. Toward wisdom-evolutionary and primitive-concise 6G:a new paradigm of semantic communication networks[J]. Engineering, 2022,8: 60-73.
[4]	QIN Z J , TAO X M , LU J H ,et al. Semantic communications:principles and challenges[J]. arXiv Preprint,arXiv:2201.01389, 2022.
[5]	石光明, 李莹玉, 谢雪梅 . 语义通讯:智能时代的产物[J]. 模式识别与人工智能, 2018,31(1): 91-99.
	SHI G M , LI Y Y , XIE X M . Semantic communications:outcome of the intelligence era[J]. Pattern Recognition and Artificial Intelligence, 2018,31(1): 91-99.
[6]	石光明, 肖泳, 李莹玉 ,等. 面向万物智联的语义通信网络[J]. 物联网学报, 2021,5(2): 26-36.
	SHI G M , XIAO Y , LI Y Y ,et al. Semantic communication networking for the intelligence of everything[J]. Chinese Journal on Internet of Things, 2021,5(2): 26-36.
[7]	NIU K , DAI J C , YAO S S ,et al. A paradigm shift toward semantic communications[J]. IEEE Communications Magazine, 2022,60(11): 113-119.
[8]	牛凯, 戴金晟, 张平 ,等. 面向6G的语义通信[J]. 移动通信, 2021,45(4): 85-90.
	NIU K , DAI J C , ZHANG P ,et al. 6G-oriented semantic communications[J]. Mobile Communications, 2021,45(4): 85-90.
[9]	DAI J C , ZHANG P , NIU K ,et al. Communication beyond transmitting bits:semantics-guided source and channel coding[J]. IEEE Wireless Communications, 2022:doi.org/10.1109/MWC.017.2100705.
[10]	XIE H Q , QIN Z J , TAO X M ,et al. Task-oriented multi-user semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022,40(9): 2584-2597.
[11]	BALLé J , CHOU P A , MINNEN D ,et al. Nonlinear transform coding[J]. IEEE Journal of Selected Topics in Signal Processing, 2020,15(2): 339-353.
[12]	RICHARDSON T , KUDEKAR S . Design of low-density parity check codes for 5G new radio[J]. IEEE Communications Magazine, 2018,56(3): 28-34.
[13]	ARIKAN E . Channel polarization:a method for constructing capacity-achieving codes for symmetric binary-input memoryless channels[J]. IEEE Transactions on Information Theory, 2009,55(7): 3051-3073.
[14]	FARSAD N , RAO M , GOLDSMITH A . Deep learning for joint source-channel coding of text[C]// Proceedings of 2018 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2018: 2326-2330.
[15]	CHOI K , TATWAWADI K , GROVER A ,et al. Neural joint source-channel coding[C]// International Conference on Machine Learning. New York:PMLR, 2019: 1182-1192.
[16]	BOURTSOULATZE E , KURKA D B , GüNDüZ D , . Deep joint source-channel coding for wireless image transmission[C]// Proceedings of 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2019: 4774-4778.
[17]	GüNDüZ D , KURKA D B , JANKOWSKI M ,et al. Communicate to learn at the edge[J]. IEEE Communications Magazine, 2020,58(12): 14-19.
[18]	KURKA D B , GüNDüZ D , . Bandwidth-agile image transmission with deep joint source-channel coding[J]. IEEE Transactions on Wireless Communications, 2021,20(12): 8081-8095.
[19]	XU J L , AI B , CHEN W ,et al. Wireless image transmission using deep source channel coding with attention modules[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022,32(4): 2315-2328.
[20]	WANG S X , YANG K , DAI J C ,et al. Distributed image transmission using deep joint source-channel coding[C]// Proceedings of 2022 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2022: 5208-5212.
[21]	WANG J , WANG S X , DAI J C ,et al. Perceptual learned source-channel coding for high-fidelity image semantic transmission[J]. arXiv Preprint,arXiv:2205.13120, 2022.
[22]	DAI J C , WANG S X , TAN K L ,et al. Nonlinear transform source-channel coding for semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022,40(8): 2300-2316.
[23]	WANG Z , SIMONCELLI E P , BOVIK A C . Multiscale structural similarity for image quality assessment[C]// Proceedings of the Thirty Seventh Asilomar Conference on Signals,Systems ＆ Computers. Piscataway:IEEE Press, 2004: 1398-1402.
[24]	ZHANG R , ISOLA P , EFROS A A ,et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 586-595.
[25]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. arXiv Preprint,arXiv:1411.1784. 2014.
[26]	DOSOVITSKIY A , BEYER L , KOLESNIKOV A ,et al. An image is worth 16×16 words:transformers for image recognition at scale[J]. arXiv Preprint,arXiv:2010.11929. 2020.
[27]	LIU Z , LIN Y T , CAO Y ,et al. Swin transformer:hierarchical vision transformer using shifted windows[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2022: 9992-10002.
[28]	BA J L , KIROS J R , HINTON G E . Layer normalization[J]. arXiv Preprint,arXiv:1607.06450. 2016.
[29]	HAN Y Z , HUANG G , SONG S J ,et al. Dynamic neural networks:a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022,44(11): 7436-7456.
[30]	XIAO Z X , YAO S S , DAI J C ,et al. Wireless deep speech semantic transmission[J]. arXiv Preprint,arXiv:2211.02283. 2022.
[31]	WANG S X , DAI J C , LIANG Z J ,et al. Wireless deep video semantic transmission[J]. IEEE Journal on Selected Areas in Communications, 2023,41(1): 214-229.
[32]	KUZNETSOVA A , ROM H , ALLDRIN N ,et al. The open images dataset v4[J]. International Journal of Computer Vision, 2020,128(7): 1956-1981.
[33]	WALLACE G K . The JPEG still picture compression standard[J]. IEEE Transactions on Consumer Electronics, 1992,38(1): 30-44.
[34]	MARCELLIN M W , GORMISH M J , BILGIN A ,et al. An overview of JPEG2000[C]// Proceedings of Data Compression Conference. Piscataway:IEEE Press, 2002: 523-541.
[35]	LIU L F , OESTGES C , POUTANEN J ,et al. The COST 2100 MIMO channel model[J]. IEEE Wireless Communications, 2012,19(6): 92-99.

面向语义通信的非线性变换编码

Nonlinear transform coding for semantic communications

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 35

相关文章 14

Metrics

推荐阅读 0

[1]	李荣鹏, 汪丙炎, 张宏纲, 赵志峰. 知识增强的语义通信接收端设计[J]. 通信学报, 2023, 44(6): 70-76.
[2]	张平, 牛凯, 姚圣时, 戴金晟. 面向未来的语义通信：基本原理与实现方法[J]. 通信学报, 2023, 44(5): 1-14.
[3]	石光明, 杨旻曦, 高大化, 柴靖轩. 面向语义信息直传的通信架构[J]. 通信学报, 2023, 44(5): 15-27.
[4]	秦志金, 赵菼菼, 李凡, 陶晓明. 多模态语义通信研究综述[J]. 通信学报, 2023, 44(5): 28-41.
[5]	江沸菠, 彭于波, 董莉. 面向6G的深度图像语义通信模型[J]. 通信学报, 2023, 44(3): 198-208.
[6]	张海君, 陈安琪, 李亚博, 隆克平. 6G移动网络关键技术[J]. 通信学报, 2022, 43(7): 189-202.
[7]	刘传宏, 郭彩丽, 杨洋, 陈九九, 朱美逸, 孙鲁楠. 面向智能任务的语义通信：理论、技术和挑战[J]. 通信学报, 2022, 43(6): 41-57.
[8]	郭红伟, 朱策, 杨栩, 罗雷. 基于失真反向传播的时域依赖率失真优化[J]. 通信学报, 2022, 43(12): 222-232.
[9]	刘传宏, 郭彩丽, 杨洋, 冯春燕, 孙启政, 陈九九. 人工智能物联网中面向智能任务的语义通信方法[J]. 通信学报, 2021, 42(11): 97-108.
[10]	李维,赵凡,张二虎,任鹏. 基于SATD的H.265/HEVC拉格朗日因子选择算法[J]. 通信学报, 2017, 38(6): 68-74.
[11]	李倩男,李云强,蒋淑静,路遥. Keccak类非线性变换的差分性质研究[J]. 通信学报, 2012, 33(9): 140-146.
[12]	郑新资,孙知信. H.264中率失真代价估计模型研究[J]. 通信学报, 2012, 33(6): 143-149.
[13]	徐进,吴志美. 结合人眼视觉特性的率失真优化帧内刷新算法[J]. 通信学报, 2007, 28(3): 35-40.
[14]	孙永梅,邱天爽. 脉冲噪声环境下的SCOT加权时间延迟估计新方法[J]. 通信学报, 2005, 26(12): 13-18.