多特征融合的合成视点立体图像质量评价

doi:10.11959/j.issn.1000-0801.2019070

电信科学 ›› 2019, Vol. 35 ›› Issue (5): 104-112.doi: 10.11959/j.issn.1000-0801.2019070

多特征融合的合成视点立体图像质量评价

崔帅南¹,彭宗举¹,邹文辉^1,²,陈芬¹,陈华¹

¹ 宁波大学信息科学与工程学院，浙江宁波 315211
² 西华师范大学电子信息工程学院，四川南充 637009

修回日期:2019-05-01 出版日期:2019-05-20 发布日期:2019-05-21
作者简介:崔帅南（1993- ），女，宁波大学信息科学与工程学院硕士生，主要研究方向为虚拟视点绘制及其质量评价。|彭宗举（1973- ），男，博士，宁波大学信息科学与工程学院教授，主要研究方向为三维视频信号处理与编码。|邹文辉（1979- ），女，宁波大学信息科学与工程学院博士生，主要研究方向为虚拟视点绘制及其质量评价。|陈芬（1973- ），女，博士，宁波大学信息科学与工程学院副教授，主要研究方向为三维视频信号处理与编码。|陈华（1988- ），男，博士，宁波大学信息科学与工程学院讲师，主要研究方向为信号与信息处理。
基金资助:
国家自然科学基金资助项目(61771269);国家自然科学基金资助项目(61620106012);国家自然科学基金资助项目(61671258);浙江省自然科学基金资助项目(LY17F010005);宁波市自然科学基金资助项目(2018A610052)

Quality assessment of synthetic viewpoint stereo image with multi-feature fusion

Shuainan CUI¹,Zongju PENG¹,Wenhui ZOU^1,²,Fen CHEN¹,Hua CHEN¹

¹ Faculty of Information Science and Engineering,Ningbo University,Ningbo 315211,China
² School of Electronics and Information Engineering,China West Normal University,Nanchong 637009,China

Revised:2019-05-01 Online:2019-05-20 Published:2019-05-21
Supported by:
The National Natural Science Foundation of China(61771269);The National Natural Science Foundation of China(61620106012);The National Natural Science Foundation of China(61671258);The Natural Science Foundation of Zhejiang Province of China(LY17F010005);The Natural Science Foundation of Ningbo of China(2018A610052)

摘要/Abstract

摘要：

提出一种多特征融合的方法，该方法融合了合成视点立体图像中失真区域、失真边缘和奇异值的特征。首先，利用视差和阈值得到符合人眼感知的失真区域，计算该区域的平均结构相似度；然后，提取合成视点边缘图像中符合人眼感知的失真边缘，计算在该区域的平均结构相似度；接着，计算原始图像与失真图像奇异值差异；最后，将上述特征值进行融合得到最终的客观质量分数。在合成视点立体图像库上进行测试，实验结果表明，Pearson 线性相关系数和Spearman相关系数均高于0.86。与现有评价方法相比，能更准确地反映合成视点立体图像质量。

关键词: 虚拟视点, 立体图像, 质量评价, 特征融合, 人眼视觉特性

Abstract:

A multi-feature fusion method was proposed,the features of the distortion areas,the distortion edges and the singular value were fused.Firstly,the distortion areas corresponding to the human eyes were obtained by using the parallax and the threshold,and then the average structural similarity in these areas was calculated.Secondly,the distorted edges in the synthetic viewpoint edge image were extracted,and then the average structural similarity was calculated.Thirdly,the difference between the original image and the distorted image singular value was calculated.Finally,the three features were fused to obtain the final objective quality score.The experiment was carried out on the synthetic viewpoint stereo image library.The experimental results show that the Pearson linear correlation coefficient and the Spearman correlation coefficient are both higher than 0.86.Compared with the existing assessment methods,it can better reflect the quality of synthetic viewpoint stereo images.

Key words: virtual viewpoint, stereo image, quality assessment, feature fusion, human visual characteristic

中图分类号:

TN391.41

崔帅南,彭宗举,邹文辉,陈芬,陈华. 多特征融合的合成视点立体图像质量评价[J]. 电信科学, 2019, 35(5): 104-112.

Shuainan CUI,Zongju PENG,Wenhui ZOU,Fen CHEN,Hua CHEN. Quality assessment of synthetic viewpoint stereo image with multi-feature fusion[J]. Telecommunications Science, 2019, 35(5): 104-112.

图/表 9

图1

图2

图3

图4

图5

图6

图7

表1

表2

参考文献 21

[1]	林金桐, 许晓东 . 第五代移动互联网[J]. 电信科学, 2015,31(5): 1-8.
	LIN J T , XU X D . The 5th generation of mobile internet[J]. Telecommunications Science, 2015,31(5): 1-8.
[2]	高峰, 王丽芳, 朱文涛 . 移动通信天线的未来技术发展[J]. 电信科学, 2016,32(Z1): 78-82.
	GAO F , WANG L F , ZHU W T . Futher technology development of mobile communication antenna[J]. Telecommunications Science, 2016,32(Z1): 78-82.
[3]	ZHI J , TAMMAM T , XIAO J ,et al. Multiview video plus depth transmission via virtual-view-assisted complementary down/upsampling[J]. Eurasip Journal on Image ＆ Video Processing, 2016(1): 19-19.
[4]	SANDIC-STANKOVIC D , KUKOLI D , CALLET P L . DIBR synthesized image quality assessment based on morphological wavelets[C]// International Workshop on Quality of Multimedia Experience,May 26-29,2015,Pylos Nestoras. Piscataway:IEEE Press, 2015: 1-4.
[5]	STANKOVIC D S , KUKOLJ D , CALLET P L . DIBR synthesized image quality assessment based on morphological pyramids[C]// 2015 International Workshop on Quality of Multimedia Experience,May 26,2015,Pylos-Nestoras,Greece. Piscataway:IEEE Press, 2015: 1-4.
[6]	MIILLERK , SCHWARZ H , MARPE D ,et al. 3D high- efficiency video coding for multi-view video and depth data[J]. IEEE Transactions on Image Processing, 2013,22(9): 3366-3378.
[7]	ZHU L , ZHANG Y , WANG S ,et al. Convolutional neural network based synthesized view quality enhancement for 3D video coding[J]. IEEE Transactions on Image Processing, 2018,27(11): 5365-5377.
[8]	TECH G , SCHWARZ H , MIILLER K ,et al. 3D video coding using the synthesized view distortion change[C]// 2012 IEEE International Conference on Multimedia and Expo,July 9-13,2012,Melbourne,VIC,Australia. Piscataway:IEEE Press, 2012: 25-28.
[9]	WANG Z , BOVIK A C , SHEIKH H R ,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004,13(4): 600-612.
[10]	BENOIT A , CALLET P L , CAMPISI P ,et al. Using disparity for quality assessment of stereoscopic images[C]// IEEE International Conference on Image Processing,Oct 12-15,2008,San Diego,CA,USA. Piscataway:IEEE Press, 2008: 389-392.
[11]	CHEN M J , SU C C , KWON D K ,et al. Full-reference quality assessment of stereopairs accounting for rivalry[J]. Signal Processing Image Communication, 2013,28(9): 1143-1155.
[12]	ZHOU Y , LI L , GU K ,et al. DIBR-synthesized image quality assessment via statistics of edge intensity and orientation[J]. Transactions on Information ＆ Systems, 2017,100(8): 1929-1933.
[13]	BATTISITI F , BOSC E , CARLI M ,et al. Objective image quality assessment of 3D synthesized views[J]. Signal Processing Image Communication, 2015,30(C): 78-88.
[14]	BOSC E , PEPION R , CALLET P L ,et al. Towards a new quality metric for 3D synthesized view assessment[J]. IEEE Journal of Selected Topics in Signal Processing, 2011,5(7): 1332-1343.
[15]	GU K , JAKHETIYA V , QIAO J F ,et al. Model-based referenceless quality metric of 3D synthesized images using local image description[J]. IEEE Transactions on Image Processing, 2018,27(1): 394-405.
[16]	TIAN S , ZHANG L , MORIN L ,et al. NIQSV+:a no-reference synthesized view quality assessment metric[J]. IEEE Transactions on Image Processing, 2018,27(4): 1652-1664.
[17]	YONG J J , KIM H G , YONG M R . Critical binocular asymmetry measure for the perceptual quality assessment of synthesized stereo 3D images in view synthesis[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2016,26(7): 1201-1214.
[18]	ZHAO Y , CHEN Z , ZHU C ,et al. Binocular just-noticeabledifference model for stereoscopic images[J]. IEEE Signal Processing Letters, 2010,18(1): 19-22.
[19]	SHANYDERMAN A , GUSEV A , ESKICIOGLU A M . An SVD-based grayscale image quality measure for local and global assessment[J]. IEEE Transactions on Image Processing, 2006,15(2): 422-429.
[20]	IVY LAB Stereoscopic 3D image database for quality assessment of synthesized stereoimages[EB]. 2014.
[21]	Middlebury stereo datasets[Z].

算法	PLCC	SROCC	RMSE
SSIM	0.351	0.381	13.585
MS_SSIM	0.637	0.647	11.180
VIF	0.388	0.299	13.373
Benoit	0.452	0.438	12.944
3DSwIM	0.527	0.499	12.330
Bosc	0.642	0.596	11.135
Chen	0.687	0.695	10.631
Jung	0.826	0.829	8.181
本文算法	0.865	0.862	12.339

序列名称	PLCC	SROCC	RMSE
Aloe	0.924	0.923	7.913
Bookarrival	0.939	0.930	6.731
Dolls	0.970	0.958	5.318
Laundry	0.918	0.930	9.988
Lovebird1	0.727	0.664	15.322
Newspaper	0.962	0.909	7.048
Reindeer	0.942	0.965	6.474

多特征融合的合成视点立体图像质量评价

Quality assessment of synthetic viewpoint stereo image with multi-feature fusion

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 21

相关文章 12

Metrics

推荐阅读 0

[1]	卢敏, 胡娟, 张先超, 丁伟健, 乐光学. 基于用户多特征融合的个性化推荐模型[J]. 电信科学, 2023, 39(5): 101-115.
[2]	胡海洋, 厉泽品, 李忠金. 特征增强和双线性特征向量融合的移动端工业货箱文本检测[J]. 电信科学, 2022, 38(7): 75-87.
[3]	邢亚芬, 殷海兵, 王鸿奎, 骆琼华. 基于视频时域感知特性的恰可察觉失真模型[J]. 电信科学, 2022, 38(2): 92-102.
[4]	林锋,徐柳婧,陈晓华,戚伟强,陈可,朱添田. 一种基于多视角特征融合的Webshell检测方法[J]. 电信科学, 2020, 36(6): 125-132.
[5]	刘汉生,唐洪玉,薄明霞,牛剑锋,李天博,李玲晓. 基于机器学习的多源威胁情报质量评价方法[J]. 电信科学, 2020, 36(1): 119-126.
[6]	王士培,彭宗举,陈芬,蒋刚毅,郁梅. 一种自由视点视频系统中立体图像质量的评价方法[J]. 电信科学, 2018, 34(3): 76-85.
[7]	贺甜甜,刘彦凯,宋利. 视频用户体验理论与实践[J]. 电信科学, 2017, 33(8): 67-76.
[8]	朱宪莹,刘箴,金炜,刘婷婷,刘翠娟,柴艳杰. 基于特征融合的层次结构微博情感分类[J]. 电信科学, 2016, 32(7): 106-114.
[9]	戴庆焰,朱仲杰. 基于图割和视域相关性的立体图像分割[J]. 电信科学, 2015, 31(11): 77-84.
[10]	罗传飞,张玮. IPTV高清频道视频质量提升研究[J]. 电信科学, 2012, 28(12): 118-123.
[11]	魏耀都,谢湘,匡镜明,黄丽. 移动视频质量评价方法及发展趋势[J]. 电信科学, 2010, 26(3): 58-63.
[12]	魏耀都,谢湘,匡镜明,黄丽. 移动视频质量评价方法及发展趋势[J]. 电信科学, 2010, 26(3): 57-63.