面向语义信息直传的通信架构

doi:10.11959/j.issn.1000-436x.2023098

通信学报 ›› 2023, Vol. 44 ›› Issue (5): 15-27.doi: 10.11959/j.issn.1000-436x.2023098

• 专题：多/跨模态语义通信 • 上一篇下一篇

面向语义信息直传的通信架构

石光明¹^,², 杨旻曦¹^,³, 高大化¹^,³, 柴靖轩¹^,³

¹ 西安电子科技大学人工智能学院, 陕西西安 710071
² 鹏城实验室，广东深圳 518055
³ 琶洲实验室，广东广州 510555

修回日期:2023-05-05 出版日期:2023-05-25 发布日期:2023-05-01
作者简介:石光明（1965- ），男，江西南昌人，博士，西安电子科技大学教授，主要研究方向为人工智能、语义通信等
杨旻曦（1996- ），男，四川成都人，西安电子科技大学博士生，主要研究方向为表征学习、计算机视觉、语义通信等
高大化（1979- ），男，河南开封人，博士，西安电子科技大学教授，主要研究方向为智能信息处理、智能感知等
柴靖轩（1996– ），男，河南郑州人，西安电子科技大学博士生，主要研究方向为知识图谱推理、语义通信、信息论等
基金资助:
国家重点研发计划基金资助项目(2019YFA0706604);国家自然科学基金资助项目(61976169);国家自然科学基金资助项目(62293483);广州市场景理解与智能交互重点实验室资助项目(202201000001)

Communication framework for directed transmission of informative semantic

Guangming SHI¹^,², Minxi YANG¹^,³, Dahua GAO¹^,³, Jingxuan CHAI¹^,³

¹ School of Artificial Intelligence, Xidian University, Xi’an 710071, China
² Peng Cheng Laboratory, Shenzhen 518055, China
³ Pazhou Lab, Guangzhou 510555, China

Revised:2023-05-05 Online:2023-05-25 Published:2023-05-01
Supported by:
The National Key Research and Development Program of China(2019YFA0706604);The National Natural Science Foundation of China(61976169);The National Natural Science Foundation of China(62293483);The Guangzhou Key Laboratory of Scene Understanding and Intelli-gent Interaction Program(202201000001)

摘要/Abstract

摘要：

随着聚焦提升带宽和频谱效率的传统通信发展模式渐入瓶颈，越来越多的研究将智能通信的目标从语法层转向语义层，通过感知并传输语义而非完整信号来节省带宽资源。信宿端智能体只需接收可理解语义中有信息部分即可。若能在信源端从语义角度中甄别出对信宿端有信息的部分加以传输，将进一步降低带宽资源和信宿端语义信息处理的时间和功耗。为此，首先探讨了智能体的语义信息处理和理解的过程，并将信宿从信息中感知到的语义划分为冗余语义、难以理解语义（暗语义）和有信息的语义（语信）；接着，提出了面向传输语义中有信息的部分的通信范式——语信通信，并将通信范式划分为语法、语义、语信、语用四层；最后，通过仿真实验验证了语信通信的可行性和有效性。这为下一代通信范式的发展提供了新思路和技术牵引。

关键词: 下一代通信, 语义通信, 语义, 信息, 通信范式分级

Abstract:

As the traditional communication development model focusing on enhancing bandwidth and spectrum efficiency is getting bottlenecked, more and more research is shifting the goal of intelligent communication from the syntactic level to the semantic level to save bandwidth resources by sensing and transmitting semantics rather than the complete signal.For the receiver agent, it is enough to receive only the information part of the understandable semantics.If the informative parts from the semantics can be filtered for transmission, it will further reduce the bandwidth resources and the time and power consumption of semantic information processing at the sender-side.To this end, firstly, the process of semantic information processing and comprehension of an intelligent body was explored.Secondly, the semantics perceived by the receiver from the message were classified into redundant semantics, unintelligible semantics, and informative semantic.Then, a communication paradigm oriented to transmitting the informative part of the semantics, called informative communication, was proposed, and the communication paradigm was extended to include four layers: syntax, semantic, informative, and pragmatic.Finally, the feasibility and effectiveness of informative communication were verified through simulation.This provides new ideas and technical traction for the development of next-generation communication models.

Key words: next-generation communication, semantic communication, semantic, information, communications paradigm hierarchy

中图分类号:

TN92

石光明, 杨旻曦, 高大化, 柴靖轩. 面向语义信息直传的通信架构[J]. 通信学报, 2023, 44(5): 15-27.

Guangming SHI, Minxi YANG, Dahua GAO, Jingxuan CHAI. Communication framework for directed transmission of informative semantic[J]. Journal on Communications, 2023, 44(5): 15-27.

图/表 14

图1

表1

图2

表2

图3

图4

图5

表3

图6

表4

图7

表5

图8

图9

参考文献 35

[1]	SHANNON C E , WEAVER W . The mathematical theory of communication[M]. Urbana: University of Illinois Press, 1949.
[2]	VORA L J . Evolution of mobile generation technology:1G to 5G and review of upcoming wireless technology 5G[J]. International journal of modern trends in engineering and research, 2015,2(10): 281-290.
[3]	ARIKAN E . Channel polarization:a method for constructing capacity-achieving codes[C]// Proceedings of 2008 IEEE International Symposium on Information Theory. Piscataway:IEEE Press, 2008: 1173-1177.
[4]	SHI G M , XIAO Y , LI Y Y ,et al. From semantic communication to semantic-aware networking:model,architecture,and open problems[J]. IEEE Communications Magazine, 2021,59(8): 44-50.
[5]	石光明, 李莹玉, 谢雪梅 . 语义通讯:智能时代的产物[J]. 模式识别与人工智能, 2018,31(1): 91-99.
	SHI G M , LI Y Y , XIE X M . Semantic communications:outcome of the intelligence era[J]. Pattern Recognition and Artificial Intelligence, 2018,31(1): 91-99.
[6]	SHI G M , GAO D H , SONG X D ,et al. A new communication paradigm:from bit accuracy to semantic fidelity[J]. arXiv Preprint,arXiv:2101.12649, 2021.
[7]	QIN Z , TAO X , LU J ,et al. Semantic communications:principles and challenges[J]. arXiv Preprint,arXiv:2201.01389, 2022.
[8]	XIE H Q , QIN Z J , LI G Y ,et al. Deep learning enabled semantic communication systems[J]. IEEE Transactions on Signal Processing, 2021,69: 2663-2675.
[9]	WENG Z Z , QIN Z J , LI G Y . Semantic communications for speech signals[C]// Proceedings of IEEE International Conference on Communications. Piscataway:IEEE Press, 2021: 1-6.
[10]	YOO H , JUNG T , DAI L ,et al. Demo:real-time semantic communications with a vision transformer[J]. arXiv Preprint,arXiv:2205.03886, 2022.
[11]	LI J G , JIA C M , ZHANG X F ,et al. Cross modal compression:towards human-comprehensible semantic compression[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York:ACM Press, 2021: 4230-4238.
[12]	HUANG D L , TAO X M , GAO F F ,et al. Deep learning-based image semantic coding for semantic communications[C]// Proceedings of IEEE Global Communications Conference (GLOBECOM). Piscataway:IEEE Press, 2022: 1-6.
[13]	HARTLEY R V L . Transmission of information[J]. The Bell System Technical Journal, 1928,7(3): 535-563.
[14]	RICHARDS A G , ASHBY W R . An introduction to cybernetics[J]. AIBS Bulletin, 1957,7(1): 41.
[15]	LONGO G . Information theory new trends and open problems[M]. Vienna: Springer Vienna, 1975.
[16]	钟义信 . 信息科学原理[M]. 福州: 福建人民出版社, 1988.
	ZHONG Y X . Principles of information science[M]. Fuzhou: Fujian People’s Publishing House, 1988.
[17]	LOWE D G . Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004,60(2): 91-110.
[18]	CRANDALL D , FELZENSZWALB P , HUTTENLOCHER D . Spatial priors for part-based recognition using statistical models[C]// Pro ceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2005: 10-17.
[19]	LOSCH M , FRITZ M , SCHIELE B . Interpretability beyond classification output:semantic bottleneck networks[J]. arXiv Preprint,arXiv:1907.10882, 2019.
[20]	JI Z L , ZOU X L , LIU X ,et al. Neural information processing in hierarchical prototypical networks[C]// International Conference on Neural Information Processing. Berlin:Springer, 2018: 603-611.
[21]	SHI G M , ZHANG Z Q , GAO D H ,et al. Knowledge-guided semantic computing network[J]. Neurocomputing, 2021,426: 70-84.
[22]	SHI G M , YANG M X , GAO D H . A novel intrinsically explainable model with semantic manifolds established via transformed priors[J]. Knowledge-Based Systems, 2022,252:109386.
[23]	石光明, 高大化, 杨旻曦 ,等. 信号的语义刻画与度量[J]. 电子学报, 2022,50(9): 2068-2078.
	SHI G M , GAO D H , YANG M X ,et al. Semantic characterization and measurement of signals[J]. Acta Electronica Sinica, 2022,50(9): 2068-2078.
[24]	KRISHNA R , ZHU Y K , GROTH O ,et al. Visual genome:connecting language and vision using crowdsourced dense image annotations[J]. International Journal of Computer Vision, 2017,123(1): 32-73.
[25]	XU D F , ZHU Y K , CHOY C B ,et al. Scene graph generation by iterative message passing[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 3097-3106.
[26]	LIN T Y , MAIRE M , BELONGIE S ,et al. Microsoft COCO:common objects in context[C]// European Conference on Computer Vision. Berlin:Springer, 2014: 740-755.
[27]	KRAUSE J , JOHNSON J , KRISHNA R ,et al. A hierarchical approach for generating descriptive image paragraphs[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 3337-3345.
[28]	DEVLIN J , CHANG M , LEE K ,et al. BERT:pre-training of deep bidirectional transformers for language understanding[J]. arXiv Preprint,arXiv:1810.04805, 2018.
[29]	TANG K H , NIU Y L , HUANG J Q ,et al. Unbiased scene graph generation from biased training[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 3713-3722.
[30]	KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv:1609.02907, 2016.
[31]	O’SHEA T , HOYDIS J . An introduction to deep learning for the physical layer[J]. IEEE Transactions on Cognitive Communications and Networking, 2017,3(4): 563-575.
[32]	ESSER P , ROMBACH R , OMMER B . Taming transformers for high-resolution image synthesis[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2021: 12868-12878.
[33]	KUZNETSOVA A , ROM H , ALLDRIN N ,et al. The open images dataset v4[J]. International Journal of Computer Vision, 2020,128(7): 1956-1981.
[34]	LESKOVEC J , KLEINBERG J , FALOUTSOS C . Graph evolution:densification and shrinking diameters[J]. ACM Transactions on Knowledge Discovery from Data, 2007,1(1): 1-42.
[35]	WANG H W , WANG J L , WANG J ,et al. Learning graph representation with generative adversarial nets[J]. IEEE Transactions on Knowledge and Data Engineering, 2021,33(8): 3090-3103.

符号	含义
	空间状态变量
	时间状态变量
	实景事物，即客观存在的事物
r_m	模态m对应的传感器
N_m	信号模态总数
R	多模态传感器集合
	模态m的实景信号，即利用r_m 从实景事物中获取的信号
	多模态实景信号集合
	模态m 的第 j 个实景语义特征函数，其中语义特征函数定义见文献[23]
	模态m下实景语义特征函数总数
	模态m下实景语义特征函数集合
	多模态实景语义特征函数集合
	信号蕴含的语义，其详细描述和刻画可见文献[23]
	虚景事物，实景事物在智能体主观中对应的虚像
	模态n下第k个虚景语义特征函数
N_n	模态n下虚景语义特征函数总数
	模态n下虚景语义特征函数集合
	多模态虚景语义特征函数集合
	模态n的虚景信号
	多模态虚景信号集合
	多模态逆虚景语义特征函数集合

符号	含义
	实景消息
	虚景消息
	多模态实景语义特征函数集合
	多模态虚景语义特征函数集合
p	根据已知虚景事物评估语义的概率，反映了接收消息中所含语义与已知虚景事物之间的符合程度
	p的高阈值
	p的低阈值
	冗余语义，满足
	暗语义，满足
	有信息的语义，简称“语信”，满足
	根据语信更新后的虚景事物

信噪比/dB	平均语义相似度
0	0.909
10	0.949
20	0.960
30	0.960

语信阈值	平均语信占比	平均语义相似度
0.30	24.92%	0.919
0.50	47.21%	0.941
0.70	63.93%	0.952
0.90	86.22%	0.962
0.99	100.00%	0.960

累计训练样本数	平均语义相似度
1	0.907
10	0.924
100	0.929
1 000	0.945
10 000	0.958

面向语义信息直传的通信架构

Communication framework for directed transmission of informative semantic

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 35

相关文章 15

Metrics

推荐阅读 0

[1]	麦伟杰, 刘伟莉, 钟竞辉. 基于种群状态信息的自适应差分进化算法[J]. 通信学报, 2023, 44(6): 34-46.
[2]	李荣鹏, 汪丙炎, 张宏纲, 赵志峰. 知识增强的语义通信接收端设计[J]. 通信学报, 2023, 44(6): 70-76.
[3]	张平, 牛凯, 姚圣时, 戴金晟. 面向未来的语义通信：基本原理与实现方法[J]. 通信学报, 2023, 44(5): 1-14.
[4]	秦志金, 赵菼菼, 李凡, 陶晓明. 多模态语义通信研究综述[J]. 通信学报, 2023, 44(5): 28-41.
[5]	房颖, 徐艺文, 赵铁松. 面向机器识别-人类感知的联合振动触觉编码[J]. 通信学报, 2023, 44(5): 42-51.
[6]	张平, 戴金晟, 张育铭, 王思贤, 秦晓琦, 牛凯. 面向语义通信的非线性变换编码[J]. 通信学报, 2023, 44(4): 1-14.
[7]	刘玉玲, 王翠林, 付章杰. 语义空间下基于情感表达的生成式文本隐写方法[J]. 通信学报, 2023, 44(4): 176-186.
[8]	江沸菠, 彭于波, 董莉. 面向6G的深度图像语义通信模型[J]. 通信学报, 2023, 44(3): 198-208.
[9]	黄冬艳, 李琨. 多地址的时间型区块链隐蔽通信方法研究[J]. 通信学报, 2023, 44(2): 148-159.
[10]	杨宏宇, 杨海云, 张良, 成翔. 基于特征依赖图的源代码漏洞检测方法[J]. 通信学报, 2023, 44(1): 103-117.
[11]	夏超, 刘亚奇, 关晴骁, 金鑫, 张艳硕, 许盛伟. 基于非线性残差的JPEG图像隐写分析[J]. 通信学报, 2023, 44(1): 142-152.
[12]	闫文浩, 姜子敬, 黄欣, 朱淑娟, 丁群. 广义二维多项式混沌映射及其在信息传输中的应用[J]. 通信学报, 2022, 43(9): 80-89.
[13]	邹晓红, 许成伟, 陈晶, 宋彪, 王明月. 大规模时序图中种子节点挖掘算法研究[J]. 通信学报, 2022, 43(9): 157-168.
[14]	李晖, 金家立, 金纾羽, 马卫娇. 基于自动选择编码及动态选词策略的文本隐写方法[J]. 通信学报, 2022, 43(9): 240-253.
[15]	陈炜宇, 骆俊杉, 王方刚, 丁海洋, 王世练, 夏国江. 无线隐蔽通信容量限与实现技术综述[J]. 通信学报, 2022, 43(8): 203-218.