基于球形加权结构相似度的全向视频编码率失真优化方法

doi:10.11959/j.issn.1000-0801.2019025

电信科学 ›› 2019, Vol. 35 ›› Issue (2): 59-69.doi: 10.11959/j.issn.1000-0801.2019025

基于球形加权结构相似度的全向视频编码率失真优化方法

周瑜锋,郁梅,杜宝祯,宋洋,蒋刚毅

宁波大学信息科学与工程学院，浙江宁波 315211

修回日期:2019-01-21 出版日期:2019-02-01 发布日期:2019-02-23
作者简介:周瑜锋（1995- ），男，宁波大学信息科学与工程学院硕士生，主要研究方向为全向视频系统信号处理与编码。|郁梅（1968- ），女，博士，宁波大学信息科学与工程学院教授，主要研究方向为三维视频系统信号处理、编码，视觉感知质量评价与系统优化。|杜宝祯（1985- ），男，宁波大学信息科学与工程学院博士生，主要研究方向为视觉感知模型研究与三维视频感知编码。|宋洋（1989- ），男，宁波大学信息科学与工程学院博士生，主要研究方向为图像处理与视觉通信、视频图像质量评价。|蒋刚毅（1964- ），男，博士，宁波大学信息科学与工程学院教授，主要研究方向为全向视频/三维视频系统信号处理、编码与传输，视觉感知质量评价与系统优化。
基金资助:
国家自然科学基金资助项目(61671258);国家自然科学基金资助项目(61871247);浙江省自然科学基金资助项目(LY15F010005)

Omni directional video coding rate distortion optimization method based on weighted-to-spherically-uniform structural similarity

Yufeng ZHOU,Mei YU,Baozhen DU,Yang SONG,Gangyi JIANG

Faculty of Information Science and Engineering,Ningbo University,Ningbo 315211,China

Revised:2019-01-21 Online:2019-02-01 Published:2019-02-23
Supported by:
The National Natural Science Foundation of China(61671258);The National Natural Science Foundation of China(61871247);The Natural Science Foundation of Zhejiang Province of China(LY15F010005)

摘要/Abstract

摘要：

针对传统视频编码中编码率失真模型的不足，考虑等矩形全向视频的特征，提出一种基于球形加权结构相似度（WS-SSIM）的全向视频编码率失真优化方法。通过考虑投影平面内部结构相似度的失真情况，以及球面失真与投影平面失真的关系，提出“球形加权结构相似度”来描述平面全向图像局部块相对于观看球面的失真程度，将其应用于全向视频编码的率失真优化过程并自适应选择量化参数，以提高基于视觉的编码效率。实验结果表明，与HEVC视频编码标准的HM16.9测试平台的方法相比，所提出方法在相同视觉质量下可平均节省24.48%的码率，说明该方法对于提高全向视频编码率失真性能具有显著的效果。

关键词: 全向视频编码, 球形加权结构相似度, 率失真优化, 量化参数自适应选择

Abstract:

Aiming at the shortcomings of the coding rate distortion model in traditional video coding,considering the characteristics of equal rectangular omnidirectional video,a method of omnidirectional video coding rate distortion optimization based on spherical weighted structural similarity (WS-SSIM) was proposed.By considering the distortion of the internal structural similarity of the projection plane and the relationship between the spherical distortion and the projection plane distortion,the “spherical weighted structural similarity” was proposed to describe the degree of distortion of the planar omnidirectional image local block relative to the viewing sphere,which was applied to the rate-distortion optimization process of omnidirectional video coding and adaptive selection of quantization parameters to improve vision-based coding efficiency.The experimental results show that compared with the HEVC video coding standard HM16.9 test platform,the proposed method can save an average of 24.48% code rate under the same visual quality,which indicates that the method has significant performance for improving the omnidirectional video coding rate distortion performance.

Key words: omnidirectional video coding, weighted-to-spherically-uniform structural similarity, rate distortion optimization, adaptive quantization parameter selection

中图分类号:

TN919

周瑜锋,郁梅,杜宝祯,宋洋,蒋刚毅. 基于球形加权结构相似度的全向视频编码率失真优化方法[J]. 电信科学, 2019, 35(2): 59-69.

Yufeng ZHOU,Mei YU,Baozhen DU,Yang SONG,Gangyi JIANG. Omni directional video coding rate distortion optimization method based on weighted-to-spherically-uniform structural similarity[J]. Telecommunications Science, 2019, 35(2): 59-69.

图/表 7

表1

图1

图2

表2

图3

表3

图4

参考文献 23

[1]	罗莹, 宋利, 解蓉 ,等. 全景媒体的系统架构研究综述[J]. 电信科学, 2018,34(2): 88-98.
	LUO Y , SONG L , XIE R ,et al. Review of system architecture of omnidirectional media[J]. Telecommunications Science, 2018,34(2): 88-98.
[2]	RAI Y , CALLET P L , GUILLOTEL P . Which saliency weighting for omni directional image quality assessment?[C]// Ninth International Conference on Quality of Multimedia Experience,May 31-June 2,2017,Erfurt,Germany. Piscataway:IEEE Press, 2017: 1-6.
[3]	CHEN Z Z , LI Y M , ZHANG Y X . Recent advances in omnidirectional video coding for virtual reality:projection and evaluation[J]. Signal Processing, 2018(146): 66-78.
[4]	罗传飞, 孔德辉, 刘翔凯 ,等. 智慧家庭的VR全景视频业务实现[J]. 电信科学, 2017,33(10): 185-193.
	LUO C F , KONG D H , LIU X K ,et al. A business realization of VR panoramic video in smart home[J]. Telecommunications Science, 2017,33(10): 185-193.
[5]	SIMONE F D , FROSSARD P , WILKINS P ,et al. Geometry-driven quantization for omnidirectional image coding[C]// Picture Coding Symposium (PCS),Dec 4-7,2016,Nuremberg,Germany. Piscataway:IEEE Press, 2016: 1-5.
[6]	LI L , LI Z , BUDAGAVI M ,et al. Projection based advanced motion model for cubic mapping for 360-degree video[C]// 2017 IEEE International Conference on Image Processing (ICIP),Sept 17-20,2017,Beijing,China. Piscataway:IEEE Press, 2017: 1427-1431.
[7]	LIN H C , HUANG C C , LI C Y ,et al. AHG8:an improvement on the compact OHP layout[C]// Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11,JVET-E0056,January 12-20,2017,Geneva,Switzerland.[S.l.:s.n. ], 2017.
[8]	ZARE A , AMINLOU A , HANNUKSELA M ,et al. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications[C]// ACM International Conference on Multimedia,October 15-19,2016,Amsterdam,The Netherlands. New York:ACM Press, 2016: 601-605.
[9]	QIAN F , GOPALAKRISHNAN V ,et al. Optimizing 360 video delivery over cellular networks[C]// The Workshop on All Things Cellular:Operations,Applications and Challenges,October 3-7,2016,New York,USA. New York:ACM Press, 2016: 1-6.
[10]	SKUPIN R , SANCHEZ Y , PODBORSKI D ,et al. HEVC tile based streaming to head mounted displays[C]// 14th IEEE Annual Consumer Communications ＆ Networking Conference (CCNC),Jan 8-11,2017,Las Vegas,NV,USA. Piscataway:IEEE Press, 2017: 613-615.
[11]	TANG M , ZHANG Y , WEN J ,et al. Optimized video coding for omnidirectional videos[C]// IEEE International Conference on Multimedia and Expo,July 10-14,2017,Hong Kong,China. Piscataway:IEEE Press, 2017: 799-804.
[12]	YOUVALARI R G , AMINLOU A , HANNUKSELA M . Analysis of regional down-sampling methods for coding of omnidirectional video[C]// Picture Coding Symposium (PCS),Dec 4-7,2016,Nuremberg,Germany. Piscataway:IEEE Press, 2016: 1-5.
[13]	HENDR Y , MUHAMMED C , MARTA K ,et al. AHG8:adaptive QP for 360o video ERP projection[C]// Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO//IEC JTC1/SC29/WG11 JVET-F0049,March 31-April 7,2017,Hobart,Australia.[S.l.:s.n. ], 2017.
[14]	SUN Y , LU A , YU L . Weighted-to-spherically-uniform quality evaluation for omnidirectional video[J]. IEEE Signal Processing Letters, 2017,24(9): 1408-1412.
[15]	YU M , LAKSHMAN H , GIROD B . A framework to evaluate omnidirectional video coding schemes[C]// IEEE International Symposium on Mixed and Augmented Reality,Sept 29-Oct 3,2015,Fukuoka,Japan. Piscataway:IEEE Press, 2015: 31-36.
[16]	ZAKHARCHENKO V , ALSHINA E , SINGH A ,et al. AHG8:suggested testing procedure for 360-degree video[C]// Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11.JVET-D0027,October 15-21,2016,Chengdu,China.[S.l.:s.n]. 2016.
[17]	WANG Z , BOVIK A C , SHEIKH H R ,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004,13(4): 600-612.
[18]	YEO C , TAN H L , TAN Y H . SSIM-based adaptive quantization in HEVC[C]// IEEE International Conference on Acoustics,Speech and Signal Processing,May 26-31,2013,Vancouver,BC,Canada. Piscataway:IEEE Press, 2013: 1690-1694.
[19]	CORBILLON X , DE SIMONE F , SIMON G . 360-degree video head movement dataset[C]// ACM on Multimedia Systems Conference,June 20-23,2017,Taipei,China. New York:ACM Press, 2017: 199-204.
[20]	ZHAO P , LIU Y , LIU J ,et al. Low-complexity content-adaptive Lagrange multiplier decision for SSIM-based RD-optimized video coding[C]// IEEE International Symposium on Circuits and Systems,May 19-23,2013,Beijing,China. Piscataway:IEEE Press, 2013: 485-488.
[21]	YEO C , TAN H L , TAN Y H . On rate distortion optimization using SSIM[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2013,23(7): 1170-1181.
[22]	SEBASTIAN S , ALIREZA A , IGOR D ,et al. AHG8:tampere pole vaulting sequence for virtual reality video coding[C]// Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11.JVET-D0143,October 15-21,2016,Chengdu,China.[S.l.:s.n]. 2016.
[23]	SUN W B , GUO R D , MEN X H ,et al. AHG8:test sequences for virtual reality video coding from LetinVR[C]// Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11.JVET-D0179,October 15-21,2016,Chengdu,China.[S.l.:s.n]. 2016.

序列名称	分辨率	位深	场景是否固定
Ballooning_vr_25p_2880×2160.yuv	4 096 dpi × 2 048 dpi	8	否
Dancing360.yuv	4 096 dpi × 2 048 dpi	8	是
DrivingInCountry_3840×1920_30fps_8bit_420_erp.yuv	3 840 dpi × 1 920 dpi	8	否
Drive360.yuv	4 096 dpi × 2 048 dpi	8	否
PoleVault_le_3840×1920_30fps_8bit_420_erp.yuv	3 840 dpi × 1 920 dpi	8	是
Paramotor_Training_vr_25p_2880×2160.yuv	4 096 dpi × 2 048 dpi	8	是

序列名称	WS-PSNR 评价模型		WS-SSIM评价模型
序列名称	BD-WS-PSNR/dB	BD-Rate	BD-WS-SSIM	BD-Rate
Balloning	0.84	-26.50%	0.008 5	-40.16%
Dancing	0.60	-14.97%	0.003 2	-27.26%
DrivingInCountry	0.41	-14.73%	0.010 1	-24.10%
Drive	0.44	-10.79%	0.002 5	-19.34%
PoleVault	0.31	-10.99%	0.005 2	-21.13%
Training	0.10	-3.65%	0.003 6	-14.90%
平均值	0.45	-13.61%	0.005 5	-24.48%

序列名称	纬度自适应QP		传统SSIM率失真优化
序列名称	BD-WS-SSIM	BD-Rate	BD-WS-SSIM	BD-Rate
Balloning	0.002 1	-8.87%	0.004 5	-22.18%
Dancing	0.002 6	-21.49%	-0.000 6	5.85%
DrivingInCountry	0.001 8	-4.38 %	0.018 5	-36.35%
Drive	0.001 0	-8.31%	0.000 3	-3.72%
PoleVault	0.001 7	-6.33%	0.020 8	-35.13%
Training	0.002 4	-7.60%	0.002 1	-9.18%
平均值	0.001 9	-9.50 %	0.007 6	-16.78%

基于球形加权结构相似度的全向视频编码率失真优化方法

Omni directional video coding rate distortion optimization method based on weighted-to-spherically-uniform structural similarity

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 23

相关文章 4

Metrics

推荐阅读 0

[1]	祁伟, 殷海兵, 王鸿奎, 黄晓峰, 牛伟宏. 基于统计建模的VVC快速码率估计算法[J]. 电信科学, 2022, 38(12): 35-45.
[2]	孟翔, 殷海兵, 黄晓峰. 基于统计建模的HEVC快速率失真估计算法[J]. 电信科学, 2021, 37(1): 58-68.
[3]	李金龙,陈芬,彭宗举,蒋刚毅,郁梅. 基于显著特征的HEVC帧内率失真优化[J]. 电信科学, 2017, 33(10): 124-133.
[4]	柴鑫刚,郑冬冬. 无线视频传输的容错编码研究[J]. 电信科学, 2011, 27(9): 89-92.