面向未来的语义通信：基本原理与实现方法

doi:10.11959/j.issn.1000-436x.2023079

通信学报 ›› 2023, Vol. 44 ›› Issue (5): 1-14.doi: 10.11959/j.issn.1000-436x.2023079

• 专题：多/跨模态语义通信 • 下一篇

面向未来的语义通信：基本原理与实现方法

张平¹, 牛凯¹, 姚圣时², 戴金晟²

¹ 北京邮电大学网络与交换技术国家重点实验室，北京 100876
² 北京邮电大学泛网无线通信教育部重点实验室，北京 100876

修回日期:2023-03-10 出版日期:2023-05-25 发布日期:2023-05-01
作者简介:张平（1959- ），男，陕西汉中人，博士，中国工程院院士，北京邮电大学教授、博士生导师，主要研究方向为先进移动通信系统等
牛凯（1976- ），男，山西文水人，博士，北京邮电大学教授、博士生导师，主要研究方向为信息论与信道编码
姚圣时（1997- ），男，江苏常熟人，北京邮电大学博士生，主要研究方向为信息论、信源信道编码和语义通信
戴金晟（1992- ），男，湖南常德人，博士，北京邮电大学副教授、博士生导师，主要研究方向为无线通信信号处理、信息论与信道编码
基金资助:
国家自然科学基金资助项目(62293481);国家自然科学基金资助项目(92067202);国家自然科学基金资助项目(62071058)

Semantic communications for future: basic principle and implementation methodology

Ping ZHANG¹, Kai NIU¹, Shengshi YAO², Jincheng DAI²

¹ The State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
² The Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China

Revised:2023-03-10 Online:2023-05-25 Published:2023-05-01
Supported by:
The National Natural Science Foundation of China(62293481);The National Natural Science Foundation of China(92067202);The National Natural Science Foundation of China(62071058)

摘要/Abstract

摘要：

阐述了语义通信的基本原理与实现方法。首先，提出了语义通信的系统模型，梳理语义通信的基本概念与术语，对比经典通信与语义通信的主要区别，总结了语义通信的技术优势。在此基础上，回顾了语义信息论的发展，构建了语义信息的度量指标体系，提出了基于归一化条件复杂度的新型度量，用于评估语义压缩的极限，探讨了语义典型序列编码的性质，揭示语义编码的渐近性能。进一步地，归纳了语义编码传输的2种典型方法——直接编码与变换编码，分析了它们的基本原理，并针对文本、语音与图像信源展示了语义编码传输的优越性能。最后，总结了语义通信的研究难点与问题，指出未来的研究方向。

关键词: 语义通信, 语义信息论, 归一化条件复杂度, 语义编码传输, 非线性变换编码

Abstract:

The basic principle of semantic communications and the associated implementation methodology were introduced.First, the systematic model of semantic communications was proposed and basic concepts and terminology were deduced by comparing the fundamental differences between classic communications and semantic communications.Thus, the technological advantages of semantic communications were concluded.Then, the development of semantic information theory was retrospected and the measurement system of semantic information was established.Furthermore, the normalized conditional complexity (NCC) was proposed to evaluate the limit of semantic compression and the property of semantic typical sequence coding was discussed so as to reveal the corresponding asymptotic performance.In addition, the semantic coding transmission methods were classified into two typical schemes, that is, direct coding and transform coding, and the basic principles of these schemes were presented.For the text, speech, and image sources, the superior performance of the semantic coding techniques were demonstrated.Finally, the difficulty and open issues of the semantic communications were further concluded and further research directions were pointed out.

Key words: semantic communications, semantic information theory, normalized conditional complexity, semantic coded transmission, nonlinear transform coding

中图分类号:

TN911.22

张平, 牛凯, 姚圣时, 戴金晟. 面向未来的语义通信：基本原理与实现方法[J]. 通信学报, 2023, 44(5): 1-14.

Ping ZHANG, Kai NIU, Shengshi YAO, Jincheng DAI. Semantic communications for future: basic principle and implementation methodology[J]. Journal on Communications, 2023, 44(5): 1-14.

图/表 15

图1

表1

表2

图2

图3

图4

图5

图6

表3

表4

图7

图8

图9

图10

图11

参考文献 69

[1]	SHANNON C E . A mathematical theory of communication[J]. Bell System Technical Journal, 1948,27(4): 623-656.
[2]	MORRIS C W . Foundations of the theory of signs[M]. Chicago: The University of Chicago Press, 1938.
[3]	SHANNON C E , WEAVER W . The mathematical theory of communication[M]. Urbana: The University of Illinois Press, 1971.
[4]	BRILLOUIN L . Science and information theory[M]. New York: Academic Press, 1956.
[5]	CARNAP R , BAR-HILLEL Y . An outline of a theory of semantic information[R]. 1952.
[6]	BAR-HILLEL Y , CARNAP R . Semantic information[J]. The British Journal for the Philosophy of Science, 1953,4(14): 147-157.
[7]	BARWISE J , PERRY J . Situations and attitudes[J]. The Journal of Philosophy, 1981,78(11): 668-691.
[8]	FLORIDI L . Outline of a theory of strongly semantic information[J]. Minds and Machines, 2004,14(2): 197-221.
[9]	KOLCHINSKY A , WOLPERT D H . Semantic information,autonomous agency and non-equilibrium statistical physics[J]. Interface Focus, 2018,8(6): 20180041.
[10]	KOUNTOURIS M , PAPPAS N . Semantics-empowered communication for networked intelligent systems[J]. IEEE Communications Magazine, 2021,59(6): 96-102.
[11]	RENYI A , . On measures of entropy and information[C]// Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability.[S.l.:s.n.], 1960: 547-561.
[12]	ZHANG P , XU W , GAO H ,et al. Toward wisdom-evolutionary and primitive-concise 6G:a new paradigm of semantic communication networks[J]. Engineering, 2022,8(1): 60-73.
[13]	ZHANG P , XU X , DONG C ,et al. Intellicise communication system:model-driven semantic communications[J]. The Journal of China Universities of Posts and Telecommunications, 2022,29(1): 2-12.
[14]	NIU K , DAI J C , YAO S S ,et al. A paradigm shift toward semantic communications[J]. IEEE Communications Magazine, 2022,60(11): 113-119.
[15]	牛凯, 戴金晟, 张平 ,等. 面向6G的语义通信[J]. 移动通信, 2021,45(4): 85-90.
	NIU K , DAI J C , ZHANG P ,et al. 6G-oriented semantic communications[J]. Mobile Communications, 2021,45(4): 85-90.
[16]	SHI G M , XIAO Y , LI Y Y ,et al. From semantic communication to semantic-aware networking:model,architecture,and open problems[J]. IEEE Communications Magazine, 2021,59(8): 44-50.
[17]	石光明, 李莹玉, 谢雪梅 . 语义通讯:智能时代的产物[J]. 模式识别与人工智能, 2018,31(1): 91-99.
	SHI G M , LI Y Y , XIE X M . Semantic communications:outcome of the intelligence era[J]. Pattern Recognition and Artificial Intelligence, 2018,31(1): 91-99.
[18]	石光明, 肖泳, 李莹玉 ,等. 面向万物智联的语义通信网络[J]. 物联网学报, 2021,5(2): 26-36.
	SHI G M , XIAO Y , LI Y Y ,et al. Semantic communication networking for the intelligence of everything[J]. Chinese Journal on Internet of Things, 2021,5(2): 26-36.
[19]	QIN Z J , TAO X M , LU J H ,et al. Semantic communications:principles and challenges[J]. arXiv Preprint,arXiv:2201.01389, 2022.
[20]	XIE H Q , QIN Z J , TAO X M ,et al. Task-oriented multi-user semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022,40(9): 2584-2597.
[21]	SHI G M , GAO D H , SONG X D ,et al. A new communication paradigm:from bit accuracy to semantic fidelity[J]. arXiv Preprint,arXiv:2101.12649, 2021.
[22]	GüNDüZ D , QIN Z J , ESTELLA A I ,et al. Guest editorial special issue on beyond transmitting bits:context,semantics,and task-oriented communications[J]. IEEE Journal on Selected Areas in Communications, 2023,41(1): 1-4.
[23]	CHACCOUR C , SAAD W , DEBBAH M ,et al. Less data,more knowledge:building next generation semantic communication networks[J]. arXiv Preprint,arXiv:2211.14343, 2022.
[24]	LUO X W , CHEN H H , GUO Q . Semantic communications:overview,open issues,and future research directions[J]. IEEE Wireless Communications, 2022,29(1): 210-219.
[25]	ZHUANG Z W , LI R , JIA K ,et al. Perception-aware multi-sensor fusion for 3D LiDAR semantic segmentation[C]// Proceedings of IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2022: 16260-16270.
[26]	MOHD T K , NGUYEN N , JAVAID A Y . Multi-modal data fusion in enhancing human-machine interaction for robotic applications:a survey[J]. arXiv Preprint,arXiv:2202.07732. 2022.
[27]	LIU F , CUI Y H , MASOUROS C ,et al. Integrated sensing and communications:toward dual-functional wireless networks for 6G and beyond[J]. IEEE Journal on Selected Areas in Communications, 2022,40(6): 1728-1767.
[28]	ZHOU F H , LI Y H , ZHANG X Y ,et al. Cognitive semantic communication systems driven by knowledge graph[C]// Proceedings of IEEE International Conference on Communications. Piscataway:IEEE Press, 2022: 4860-4865.
[29]	ZHOU K , ZHAO W X , BIAN S Q ,et al. Improving conversational recommender systems via knowledge graph based semantic fusion[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining. New York:ACM Press, 2020: 1006-1014.
[30]	LIU P F , YUAN W Z , FU J L ,et al. Pre-train,prompt,and predict:a systematic survey of prompting methods in natural language processing[J]. ACM Computing Surveys, 2023,55(9): 1-35.
[31]	LI W B , LIN Z , ZHOU K ,et al. MAT:mask-aware transformer for large hole image inpainting[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2022: 10748-10758.
[32]	ZHANG R , ISOLA P , EFROS A A ,et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 586-595.
[33]	RIX A W , BEERENDS J G , HOLLIER M P ,et al. Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs[C]// Proceedings of 2001 IEEE International Conference on Acoustics,Speech,and Signal Processing. Piscataway:IEEE Press, 2002: 749-752.
[34]	BAO J , BASU P , DEAN M K ,et al. Towards a theory of semantic communication[C]// Proceedings of 2011 IEEE Network Science Workshop. Piscataway:IEEE Press, 2011: 110-117.
[35]	LUCA A D , TERMINI S . A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory[J]. Information and Control, 1972,20(4): 301-312.
[36]	LUCA A D , TERMINI S . Entropy of L-fuzzy sets[J]. Information and Control, 1974,24(1): 55-73.
[37]	K?RNER J , . Coding of an information source having ambiguous alphabet and the entropy of graphs[C]// Proceeding of the 6th Prague Conference on Information Theory.[S.l.:s.n.], 1973: 411-425.
[38]	KORNER J , MARTON K . Random access communication and graph entropy[J]. IEEE Transactions on Information Theory, 1988,34(2): 312-314.
[39]	ALON N , ORLITSKY A . Source coding and graph entropies[J]. IEEE Transactions on Information Theory, 1996,42(5): 1329-1339.
[40]	吴伟陵 . 广义信源与广义熵[J]. 北京邮电大学学报, 1982,5(1): 29-41.
	WU W L . Generalized information source and generalized entropy[J]. Journal of Beijing University of Posts and Telecommunications, 1982,5(1): 29-41.
[41]	钟义信 . 信息科学原理[M]. 北京: 北京邮电大学出版社, 1996.
	ZHONG Y X . Theory of information science[M]. Beijing: Beijing University of Posts and Telecommunications Press, 1996.
[42]	钟义信 . 自然语言理解的全信息方法论[J]. 北京邮电大学学报, 2004,27(4): 1-12.
	ZHONG Y X . Comprehensive information based methodology for natural language understanding[J]. Journal of Beijing University of Posts and Telecommunications, 2004,27(4): 1-12.
[43]	钟义信 . 信息生态学与语义信息论[J]. 图书情报知识, 2017(6): 4-11.
	ZHONG Y X . Information ecology and semantic information theory[J]. Document,Information ＆ Knowledge, 2017(6): 4-11.
[44]	鲁晨光 . 广义熵和广义互信息的编码意义[J]. 通信学报, 1994,15(6): 37-44.
	LU C G . Meanings of generalized entropy and generalized mutual information for coding[J]. Journal of China Institute of Communications, 1994,15(6): 37-44.
[45]	FAN P Y , DONG Y Q , LU J X ,et al. Message importance measure and its application to minority subset detection in big data[C]// Proceedings of 2016 IEEE GLOBECOM Workshops. Piscataway:IEEE Press, 2017: 1-5.
[46]	SHE R , LIU S Y , FAN P Y . Recognizing information feature variation:message importance transfer measure and its applications in big data[J]. Entropy, 2018,20(6): 401.
[47]	LI M , VITáNYI P , . An introduction to Kolmogorov complexity and its applications[M]. New York: Springer, 2008.
[48]	BOURTSOULATZE E , BURTH K D , GüNDüZ D , . Deep joint source-channel coding for wireless image transmission[J]. IEEE Transactions on Cognitive Communications and Networking, 2019,5(3): 567-579.
[49]	TUNG T Y , KURKA D B , JANKOWSKI M ,et al. DeepJSCC-Q:channel input constrained deep joint source-channel coding[C]// Proceedings of IEEE International Conference on Communications. Piscataway:IEEE Press, 2022: 3880-3885.
[50]	XIE H Q , QIN Z J , LI G Y ,et al. Deep learning enabled semantic communication systems[J]. IEEE Transactions on Signal Processing, 2021,69: 2663-2675.
[51]	WENG Z Z , QIN Z J . Semantic communication systems for speech transmission[J]. IEEE Journal on Selected Areas in Communications, 2021,39(8): 2434-2444.
[52]	XIE H Q , QIN Z J . A lite distributed semantic communication system for Internet of things[J]. IEEE Journal on Selected Areas in Communications, 2021,39(1): 142-153.
[53]	BALLé J , CHOU P A , MINNEN D ,et al. Nonlinear transform coding[J]. IEEE Journal of Selected Topics in Signal Processing, 2020,15(2): 339-353.
[54]	BALLé J , LAPARRA V , SIMONCELLI E P . End-to-end optimized image compression[J]. arXiv Preprint,arXiv:1611.01704v3, 2016.
[55]	YAO S S , XIAO Z X , DAI J C ,et al. Variational speech waveform compression to catalyze semantic communications[C]// Proceedings of IEEE Wireless Communications and Networking Conference. Piscataway:IEEE Press, 2023: 1-6.
[56]	DAI J C , ZHANG P , NIU K ,et al. Communication beyond transmitting bits:semantics-guided source and channel coding[J]. IEEE Wireless Communications, 2022:doi.org/10.1109/mwc.017.2100705.
[57]	DAI J C , WANG S X , TAN K L ,et al. Nonlinear transform source-channel coding for semantic communications[J]. IEEE Journal on Selected Areas in Communications, 2022,40(8): 2300-2316.
[58]	WANG S X , DAI J C , LIANG Z J ,et al. Wireless deep video semantic transmission[J]. IEEE Journal on Selected Areas in Communications, 2022,41(1): 214-229.
[59]	WANG J , WANG S X , DAI J C ,et al. Perceptual learned source-channel coding for high-fidelity image semantic transmission[C]// Proceedings of 2022 Global Communications Conference. Piscataway:IEEE Press, 2022: 3959-3964.
[60]	XIAO Z X , YAO S S , DAI J C ,et al. Wireless deep speech semantic transmission[C]// Proceedings of 2023 International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2023: 1-5.
[61]	YAO S X , WANG S X , DAI J C ,et al. Semantic information processing in industrial networks[J]. The Journal of China Universities of Posts and Telecommunications, 2022,29(1): 41-49.
[62]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York:ACM Press, 2017: 6000-6010.
[63]	WANG S X , DAI J C , YAO S S ,et al. A novel deep learning architecture for wireless image transmission[C]// Proceedings of 2021 IEEE Global Communications Conference. Piscataway:IEEE Press, 2022: 1-6.
[64]	YAO S S , NIU K , WANG S X ,et al. Semantic coding for text transmission:an iterative design[J]. IEEE Transactions on Cognitive Communications and Networking, 2022,8(4): 1594-1603.
[65]	LIU Z , LIN Y T , CAO Y ,et al. Swin transformer:hierarchical vision transformer using shifted windows[C]// Proceedings of IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2022: 9992-10002.
[66]	LIU L F , OESTGES C , POUTANEN J ,et al. The COST2100 MIMO channel model[J]. IEEE Wireless Communications, 2012,19(6): 92-99.
[67]	BESSETTE B , SALAMI R , LEFEBVRE R ,et al. The adaptive multirate wideband speech codec (AMR-WB)[J]. IEEE Transactions on Speech and Audio Processing, 2002,10(8): 620-636.
[68]	VALIN J , VOS K , TERRIBERRY T B . Definition of the opus audio codec[J]. RFC, 2012,6716: 1-326.
[69]	SCHOEFFLER M , ST?TER F R , EDLER B . Towards the next generation of web-based experiments:a case study assessing basic audio quality following the ITU-R recommendation BS.1534 (MUSHRA)[C]// Proceedings of the 1st Web Audio Conference.[S.l.:s.n.], 2015: 1-6.

通信系统	作用域	处理对象	先验知识	处理方法	性能准则	优化目标
语义通信系统	语义空间	语义基等语义特征	知识库	语义特征提取、语义内容识别、多模态语义融合、语义纠错、语义非平衡传输	端到端语义误差，包括感知质量指标、任务执行度（效用分数）等	速率-效用函数
经典通信系统	信号空间	比特流	无	信源编码、信道编码、链路自适应技术、多载波传输技术、多址接入、多天线技术	最小均方误差（MMSE）等	错误率（误比特率、误块率）

指标	含义	表达式
语义熵	平均语义信息量
语义互信息	一个对象包含关于另一个对象的语义信息量
平均语义失真	语义通信导致的语义信息损失
语义信道容量	特定语义失真下的最大传输速率
语义率失真函数	特定语义失真下的最小编码速率函数

编码方案	文本语句相似度
Huffman编码级联RS码	0.03
Huffman编码级联Turbo码	0.82
DeepSC方案	0.96

编码方案	PESQ
PCM编码级联Turbo码	2.04
直接语义编码传输方案	4.3

面向未来的语义通信：基本原理与实现方法

Semantic communications for future: basic principle and implementation methodology

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 69

相关文章 8

Metrics

推荐阅读 0

[1]	李荣鹏, 汪丙炎, 张宏纲, 赵志峰. 知识增强的语义通信接收端设计[J]. 通信学报, 2023, 44(6): 70-76.
[2]	石光明, 杨旻曦, 高大化, 柴靖轩. 面向语义信息直传的通信架构[J]. 通信学报, 2023, 44(5): 15-27.
[3]	秦志金, 赵菼菼, 李凡, 陶晓明. 多模态语义通信研究综述[J]. 通信学报, 2023, 44(5): 28-41.
[4]	张平, 戴金晟, 张育铭, 王思贤, 秦晓琦, 牛凯. 面向语义通信的非线性变换编码[J]. 通信学报, 2023, 44(4): 1-14.
[5]	江沸菠, 彭于波, 董莉. 面向6G的深度图像语义通信模型[J]. 通信学报, 2023, 44(3): 198-208.
[6]	张海君, 陈安琪, 李亚博, 隆克平. 6G移动网络关键技术[J]. 通信学报, 2022, 43(7): 189-202.
[7]	刘传宏, 郭彩丽, 杨洋, 陈九九, 朱美逸, 孙鲁楠. 面向智能任务的语义通信：理论、技术和挑战[J]. 通信学报, 2022, 43(6): 41-57.
[8]	刘传宏, 郭彩丽, 杨洋, 冯春燕, 孙启政, 陈九九. 人工智能物联网中面向智能任务的语义通信方法[J]. 通信学报, 2021, 42(11): 97-108.