基于特征融合的通信语音干扰效果客观评估

doi:10.11959/j.issn.1000-436x.2023043

通信学报 ›› 2023, Vol. 44 ›› Issue (3): 105-116.doi: 10.11959/j.issn.1000-436x.2023043

基于特征融合的通信语音干扰效果客观评估

林云¹, 徐怀韬¹, 王森¹, 张思成¹, 庄龙²

¹ 哈尔滨工程大学信息与通信工程学院，黑龙江哈尔滨 150001
² 安徽大学集成电路学院，安徽合肥 230039

修回日期:2022-11-30 出版日期:2023-03-25 发布日期:2023-03-01
作者简介:林云（1980− ），男，黑龙江哈尔滨人，哈尔滨工程大学教授、博士生导师，主要研究方向为智能无线电技术、人工智能和机器学习、大数据分析与挖掘、软件和认知无线电、信息安全与对抗、智能信息处理
徐怀韬（1998− ），男，江西南昌人，哈尔滨工程大学硕士生，主要研究方向为通信干扰语音质量评估
王森（1994− ），男，吉林四平人，哈尔滨工程大学博士生，主要研究方向为干扰评估、无线通信及信号处理
张思成（1996− ），男，山东临沂人，哈尔滨工程大学博士生，主要研究方向为基于深度学习的智能电磁信号处理
庄龙（1998− ），男，江苏徐州人，安徽大学硕士生，主要研究方向为雷达信号处理和计算机视觉
基金资助:
国家自然科学基金资助项目(62201172);中央高校基本科研业务费专项资金资助项目(3072022CF0804);中央高校基本科研业务费专项资金资助项目(3072022CF0601)

Objective assessment of communication speech interference effect based on feature fusion

Yun LIN¹, Huaitao XU¹, Sen WANG¹, Sicheng ZHANG¹, Long ZHUANG²

¹ College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China
² School of Integrated Circuits, Anhui University, Hefei 230039, China

Revised:2022-11-30 Online:2023-03-25 Published:2023-03-01
Supported by:
The National Natural Science Foundation of China(62201172);The Fundamental Research Funds for the Central Universities(3072022CF0804);The Fundamental Research Funds for the Central Universities(3072022CF0601)

摘要/Abstract

摘要：

针对通信语音干扰效果客观评估问题，提出了基于多测度与多模态融合的2种评估方法。首先，利用端点检测算法以及动态时间弯折算法对受扰语音数据进行预处理。然后，提取数据中的语音内容并与标准语音进行测度计算得到5种测度，将5种测度融合后利用随机森林模型进行质量等级评估。最后，结合多模态融合技术，设计了基于残差结构的神经网络模型，融合受扰语音数据的图域、测度域特征并进行质量等级评估。实验结果表明，2种方法的评估准确率均达到了90%以上。其中，多模态评估方法与现有的研究方法相比，准确率提升了约3.269%，证明所提方法具有更优的性能。

关键词: 语音质量评估, 语音信号处理, 多模态融合, 深度神经网络

Abstract:

In view of the objective assessment problem of the effect of communication speech interference, methods based on multi-measurements and multimodal fusion were proposed.First, the interfered speech was preprocessed by the endpoint detection algorithm and time warping algorithm.Then, the content of speech was extracted and performed measurement calculated with the standard speech to obtain five kinds of measure.After the fusion of five measures, random forest model was used to assessed the quality level.Finally, a neural network model based on residual structure was designed combined multimodal fusion technique, which fused the graph domain and measure domain features of the interfered speech data and performed quality level assessment.Experimental results show that the accuracy of two methods have reached more than 90%.Among them, the multimodal assessment method improves the accuracy by about 3.269% compared with the existing research methods, which proves that it has a better performance.

Key words: speech quality assessment, speech signal processing, multimodal fusion, deep neural network

中图分类号:

TN912.3

林云, 徐怀韬, 王森, 张思成, 庄龙. 基于特征融合的通信语音干扰效果客观评估[J]. 通信学报, 2023, 44(3): 105-116.

Yun LIN, Huaitao XU, Sen WANG, Sicheng ZHANG, Long ZHUANG. Objective assessment of communication speech interference effect based on feature fusion[J]. Journal on Communications, 2023, 44(3): 105-116.

图/表 21

表1

图1

图2

表2

表3

图3

图4

图5

表4

图6

图7

图8

图9

图10

图11

图12

图13

表5

图14

图15

表6

分类器及模型Macro-F1分数"

分类器	Macro-F1
支持向量机	0.88
逻辑回归	0.89
随机森林	$0 . 90$
AlexNet	0.88
VGG16	0.89
残差结构	$0 . 91$

表6

参考文献 25

[1]	潘志丽, 张宏科, 张思东 . 现代电子干扰理论与效能评估的研究[J]. 通信学报, 2003,24(11): 40-45.
	PAN Z L , ZHANG H K , ZHANG S D . Research on modern electronic jamming theory and efficiency evaluation[J]. Journal of China Institute of Communications, 2003,24(11): 40-45.
[2]	ITU-T. Mean opinion score（MOS）terminology[S]. 2003.
[3]	RIX A W , BEERENDS J G , HOLLIER M P ,et al. Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs[C]// Proceedings of 2001 IEEE International Conference on Acoustics,Speech,and Signal Processing. Piscataway:IEEE Press, 2002: 749-752.
[4]	BEERENDS J G , SCHMIDMER C , BERGER J ,et al. Perceptual objective listening quality assessment (POLQA),the third generation ITU-T standard for end-to-end speech quality measurement part I—temporal alignment[J]. Journal of the Audio Engineering Society, 2013,61(6): 366-384.
[5]	AFFONSO E T , ROSA R L , RODRíGUEZ D Z , . Speech quality assessment over lossy transmission channels using deep belief networks[J]. IEEE Signal Processing Letters, 2018,25(1): 70-74.
[6]	FU S W , TSAO Y , HWANG H T ,et al. Quality-net:an end-to-end non-intrusive speech quality assessment model based on BLSTM[J]. arXiv Preprint,arXiv:1808.05344, 2018.
[7]	FU S W , LIAO C F , TSAO Y . Learning with learned loss function:speech enhancement with quality-net to improve perceptual evaluation of speech quality[J]. IEEE Signal Processing Letters, 2020,27: 26-30.
[8]	RODRíGUEZ D Z , PíVARO G F , ROSA R L ,et al. Improving a parametric model for speech quality assessment in wireless communication systems[C]// Proceedings of 2018 26th International Conference on Software,Telecommunications and Computer Networks. Piscataway:IEEE Press, 2018: 1-5.
[9]	ITU-T. The E-model:a computational model for use in transmission planning:G.107[S]. 2002.
[10]	LO C C , FU S W , HUANG W C ,et al. MOSNet:deep learning based objective assessment for voice conversion[J]. arXiv Preprint,arXiv:1904.08352, 2019.
[11]	ZHANG L , ZHAO X L , LI X . Assessment of extreme communication environment with ultralow SNR:a benchmark[J]. IEEE Access, 2021,9: 45400-45406.
[12]	WANG S , LIN Y , HAO M ,et al. Interference quality assessment of speech communication based on deep learning[J]. IEEE Transactions on Reliability, 2022,71(2): 1011-1021.
[13]	HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 770-778.
[14]	龙华, 杨明亮, 邵玉斌 . 基于特征流融合的带噪语音检测算法[J]. 通信学报, 2020,41(4): 134-142.
	LONG H , YANG M L , SHAO Y B . Noisy voice detection algorithm based on feature stream fusion[J]. Journal on Communications, 2020,41(4): 134-142.
[15]	张璐琳, 张磊, 赵凌伟 . 语音通信干扰效果评定规则:GJB4405B-2017[S]. 2017.
	ZHANG L L , ZHANG L , ZHAO L W . Speech communication interference effect assessment rules:GJB4405B-2017[S]. 2017.
[16]	傅恒丰 . 语音通信干扰效果评估方法研究[D]. 哈尔滨:哈尔滨工程大学, 2021.
	FU H F . Research on the evaluation method of speech communication interference effect[D]. Harbin:Harbin Engineering University, 2021.
[17]	DUBEY R K , KUMAR A . Non-intrusive objective speech quality assessment using a combination of MFCC,PLP and LSF features[C]// Proceedings of 2013 International Conference on Signal Processing and Communication. Piscataway:IEEE Press, 2014: 297-302.
[18]	张文克 . 融合LPCC和MFCC特征参数的语音识别技术的研究[D]. 湘潭:湘潭大学, 2016.
	ZHANG W K . The research of fusion LPCC and MFCC feature parameters in speech recognition technology[D]. Xiangtan:Xiangtan University, 2016.
[19]	KEERTHANA Y M , REDDY M K , RAO K S . CWT-based approach for epoch extraction from telephone quality speech[J]. IEEE Signal Processing Letters, 2019,26(8): 1107-1111.
[20]	杨路飞, 章新华, 吴秉坤 ,等. 基于 MFCC特征的被动水声目标深度学习分类方法[J]. 舰船科学技术, 2020,42(19): 129-133.
	YANG L F , ZHANG X H , WU B K ,et al. Research on the classification method of passive acoustic target depth learning based on MFCC[J]. Ship Science and Technology, 2020,42(19): 129-133.
[21]	MUDA L , BEGAM M , ELAMVAZUTHI I . Voice recognition algorithms using Mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques[J]. arXiv Preprint,arXiv:1003.4083, 2010.
[22]	TIN K H . The random subspace method for constructing decision forests[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998,20(8): 832-844.
[23]	MITTAG G , NADERI B , CHEHADI A ,et al. NISQA:a deep CNN-self-attention model for multidimensional speech quality prediction with crowdsourced datasets[J]. arXiv Preprint,arXiv:2104.09494, 2021.
[24]	YU M , ZHANG C , XU Y ,et al. MetricNet:towards improved modeling for non-intrusive speech quality assessment[J]. arXiv Preprint,arXiv:2104.01227, 2021.
[25]	张思成, 林云, 涂涯 ,等. 基于轻量级深度神经网络的电磁信号调制识别技术[J]. 通信学报, 2020,41(11): 12-21.
	ZHANG S C , LIN Y , TU Y ,et al. Electromagnetic signal modulation recognition technology based on lightweight deep neural network[J]. Journal on Communications, 2020,41(11): 12-21.

数码	读音
0	Dong（洞）
1	Yao（幺）
2	Liang（两）
3	San（三）
4	Si（四）
5	Wu（五）
6	Liu（六）
7	Guai（拐）
8	Ba（八）
9	Gou（勾）

干扰参数	值
干扰样式	噪声调频
干扰频点/MHz	88.2
干扰带宽/MHz	1
干扰功率/W	1

信息损伤级G_s	r	语音质量描述
1级	100%≥r＞ 95%	干扰噪声弱，字音较清晰，个别单字不能准确判别，通信内容可懂
2级	95%≥r＞ 80%	干扰噪声较强，部分字音较不清晰，个别单字不能准确判别，通信内容基本可懂
3级	80%≥r＞ 50%	干扰噪声强，字音不清晰，大部分单字不能判别，通信内容基本不可懂
4级	50%≥r＞ 10%	干扰噪声强，字音模糊，大部分单字不能判别，通信内容不可懂
5级	10%≥r≥0	干扰噪声很强，绝大部分单字不能准确判别，通信内容完全不可懂

特征类型	特征性质
梅尔特征（MFCC、梅尔能量）	听觉模型
LPC、LPCC	声道模型
小波统计特征	时频分析模型

分类器	单一测度	平均准确率
	MFCC	87.18%
	梅尔能量	88.21%
支持向量机	LPC	80.51%
	LPCC	81.54%
	小波统计特征	83.08%
	MFCC	80.00%
	梅尔能量	84.62%
逻辑回归	LPC	75.90%
	LPCC	77.44%
	小波统计特征	79.49%
	MFCC	85.13%
	梅尔能量	89.74%
随机森林	LPC	86.15%
	LPCC	81.03%
	小波统计特征	82.56%

基于特征融合的通信语音干扰效果客观评估

Objective assessment of communication speech interference effect based on feature fusion

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 21

参考文献 25

相关文章 8

Metrics

推荐阅读 0

[1]	熊金波, 周永洁, 毕仁万, 万良, 田有亮. 边缘协同的轻量级隐私保护分类框架[J]. 通信学报, 2022, 43(1): 127-137.
[2]	王哲, 李陶深, 葛丽娜, 张桂芬, 吴敏. 基于深度学习的传感云sink节点最优能效SWIPT波束成形设计[J]. 通信学报, 2021, 42(7): 176-188.
[3]	傅友华, 陈栋. 混合智能反射表面结构辅助的毫米波通信信道估计[J]. 通信学报, 2021, 42(10): 189-196.
[4]	吴伟华,柴冠华,杨清海,刘润滋. 面向不确定CSI随机接入网络的深度稳健资源分配[J]. 通信学报, 2020, 41(7): 29-37.
[5]	张思成,林云,涂涯,ShiwenMao. 基于轻量级深度神经网络的电磁信号调制识别技术[J]. 通信学报, 2020, 41(11): 12-21.
[6]	胡铮,袁浩,朱新宁,倪万里. 面向5G需求的人群流量预测模型研究[J]. 通信学报, 2019, 40(2): 1-10.
[7]	桂冠,王禹,黄浩. 基于深度学习的物理层无线通信技术：机遇与挑战[J]. 通信学报, 2019, 40(2): 19-23.
[8]	居太亮,彭启琮,邵怀宗,林静然. 基于任意麦克风阵列的声源二维DOA估计算法研究[J]. 通信学报, 2005, 26(8): 129-133.