视频主观观测实验启发的HEVC感知帧内码率控制

doi:10.11959/j.issn.1000-436x.2021146

通信学报 ›› 2021, Vol. 42 ›› Issue (8): 90-102.doi: 10.11959/j.issn.1000-436x.2021146

视频主观观测实验启发的HEVC感知帧内码率控制

公衍超¹^,², 王玲¹, 刘颖¹^,², 杨楷芳³, 林庆帆¹^,⁴, 王富平¹^,²

¹ 西安邮电大学通信与信息工程学院，陕西西安 710121
² 无线通信与信息处理技术国际联合研究中心，陕西西安 710121
³ 陕西师范大学计算机科学学院，陕西西安 710119
⁴ 新加坡Maschine Technology有限公司，新加坡 787820

修回日期:2021-05-06 出版日期:2021-08-25 发布日期:2021-08-01
作者简介:公衍超（1986- ），男，山东蒙阴人，博士，西安邮电大学讲师、硕士生导师，主要研究方向为视频编码、视频质量评价
王玲（1994- ），女，河南光山人，西安邮电大学硕士生，主要研究方向为视频编码
刘颖（1972- ），女，陕西户县人，博士，西安邮电大学教授、博士生导师，主要研究方向为图像检索、图像识别与理解
杨楷芳（1987- ），女，山西阳泉人，博士，陕西师范大学副教授、硕士生导师，主要研究方向为视频编码、视频质量评价
林庆帆（1969- ），男，福建晋江人，博士，西安邮电大学特聘教授，主要研究方向为视频编码、图像清晰化及人工智能
王富平（1987- ），男，陕西咸阳人，博士，西安邮电大学讲师、硕士生导师，主要研究方向为图像特征提取
基金资助:
国家自然科学基金资助项目(61801381);国家自然科学基金资助项目(61801282);陕西省自然科学基础研究基金资助项目(2020JQ-426);陕西省国际科技合作与交流计划基金资助项目(2018KW-003);陕西省教育厅科研计划基金资助项目(18JK0708);西安邮电大学研究生联合培养工作站重点基金资助项目(YJGJ201902)

HEVC perceptual intra-frame rate control inspired by video subjective observation experiment

Yanchao GONG¹^,², Ling WANG¹, Ying LIU¹^,², Kaifang YANG³, Qingfan LIN¹^,⁴, Fuping WANG¹^,²

¹ School of Communications and Information Engineering, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
² International Joint Research Center for Wireless Communication and Information Processing Technology, Xi’an 710121, China
³ School of Computer Science, Shaanxi Normal University, Xi’an 710119, China
⁴ Maschine Technology Pte.Ltd., 787820, Singapore

Revised:2021-05-06 Online:2021-08-25 Published:2021-08-01
Supported by:
The National Natural Science Foundation of China(61801381);The National Natural Science Foundation of China(61801282);The Project of Natural Science Basic Research in Shaanxi Province(2020JQ-426);The Project of International Scientific and Technological Cooperation and Exchange in Shaanxi Province(2018KW-003);The Project of Education Department of Shaanxi Province(18JK0708);The Key Project of Postgraduate Joint Training Workstation of Xi’an University of Posts and Telecommunications(YJGJ201902)

摘要/Abstract

摘要：

针对高效视频编码（HEVC）采用的码率控制方法感知率失真性能较低的问题，提出了一种有效提高视频感知率失真性能的帧内码率控制算法。首先，度量视频的空时域复杂度，并结合空时域复杂度设计视频主观观测实验。其次，基于视频主观观测实验结果，结合人类视觉系统相关感知特性，构建能够有效衡量人眼对视频不同区域感知差异性的视频内容空时域感知敏感因子。最后，将空时域感知敏感因子运用到帧内图像最大编码单元层目标比特分配中，实现感知帧内码率控制。实验结果表明，在比特估计准确度和感知率失真性能2个核心指标上，所提算法均明显优于HEVC采用的码率控制算法。

关键词: 高效视频编码, 码率控制, 人类视觉感知, 目标比特分配

Abstract:

In view of the low perceptual rate-distortion performance of the rate control method used in high efficiency video coding (HEVC), an intra-frame rate control algorithm to effectively improve the perceptual rate-distortion performance of video was proposed.First, the spatiotemporal complexity of videos was measured, and the video subjective observation experiment was designed combined with the spatiotemporal complexity.Secondly, based on the experimental results of video subjective observation, combined with the relevant human vision system perception characteristics, the spatiotemporal perceptual sensitivity factors of video content which could effectively measure the perception difference of human eyes for different regions of video were constructed.Finally, the spatiotemporal perceptual sensitivity factors were applied to the target bit allocation of the largest coding unit layer in the intra-frame image, then the perceptual intra-frame rate control was realized.Experimental results show that the proposed algorithm is superior to the rate control algorithm adopted by HEVC in terms of the bit estimation accuracy and the perceptual rate-distortion performance.

Key words: high efficiency video coding, rate control, human vision perception, target bit allocation

中图分类号:

TN911.7

公衍超, 王玲, 刘颖, 杨楷芳, 林庆帆, 王富平. 视频主观观测实验启发的HEVC感知帧内码率控制[J]. 通信学报, 2021, 42(8): 90-102.

Yanchao GONG, Ling WANG, Ying LIU, Kaifang YANG, Qingfan LIN, Fuping WANG. HEVC perceptual intra-frame rate control inspired by video subjective observation experiment[J]. Journal on Communications, 2021, 42(8): 90-102.

图/表 14

图1

图2

图3

图4

图5

表1

表2

图6

图7

图8

图9

图10

图11

表3

参考文献 38

[1]	LI B , LI H Q , LI L ,et al. λ domain rate control algorithm for high efficiency video coding[J]. IEEE Transactions on Image Processing, 2014,23(9): 3841-3854.
[2]	SULLIVAN G J , OHM J R , HAN W J ,et al. Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012,22(12): 1649-1668.
[3]	GONG Y C , WAN S , YANG K F ,et al. [J]. Temporal-layer-motivated lambda domain picture level rate control for random-access configuration in H.265/HEVC, 2019,29(1): 156-170.
[4]	ZHAO F , KU C W , XIANG G ,et al. A novel quality enhanced low complexity rate control algorithm for HEVC[C]// 2020 IEEE International Conference on Visual Communications and Image Processing. Piscataway:IEEE Press, 2020: 278-280.
[5]	LI B , LI H Q , LI L ,et al. Rate control by R-lambda model for HEVC:JCTVC-K0103[C]// Proceedings of the 11th Joint Collaborative Team on Video Coding Meeting. New York:ACM Press, 2012: 1-11.
[6]	WANG X , KARCZEWICZ M . Intra frame rate control based SATD:JCT-VC M0257[C]// Proceedings of the 13th Joint Collaborative Team on Video Coding Meeting. New York:ACM Press, 2013: 1-5.
[7]	LI B , LI H Q , LI L ,et al. Adaptive bit allocation for R-lambda model rate control in HM:JCTVC-M0036[C]// Proceedings of the 13th Joint Collaborative Team on Video Coding Meeting. New York:ACM Press, 2013: 1-7.
[8]	LI L , LI B , LI H Q ,et al. λ-domain optimal bit allocation algorithm for high efficiency video coding[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018,28(1): 130-142.
[9]	李维, 赵凡, 张二虎 ,等. 基于SATD的H.265/HEVC拉格朗日因子选择算法[J]. 通信学报, 2017,38(6): 68-74.
	LI W , ZHAO F , ZHANG E H ,et al. Lagrange multiplier selection based on SATD for H.265/HEVC[J]. Journal on Communications, 2017,38(6): 68-74.
[10]	MCCANN K , ROSEWARNE C , BROSS B ,et al. High efficiency video coding (HEVC) test model 16 (HM 16) encoder description:JCTVC-R1002[C]// Proceedings of the 18th Joint Collaborative Team on Video Coding Meeting. New York:ACM Press, 2014: 1-59.
[11]	LI X J , ABUDOULIKEMU Y . Background-foreground information based bit allocation algorithm for surveillance video on high efficiency video coding (HEVC)[C]// 2016 IEEE International Conference on Visual Communications and Image Processing. Piscataway:IEEE Press, 2016: 1-4.
[12]	ZHANG Z W , JING T , HAN J N ,et al. A new rate control scheme for video coding based on region of interest[J]. IEEE Access, 2017,5: 13677-13688.
[13]	WANG M H , NGAN K N , LI H L . An efficient frame-content based intra frame rate control for high efficiency video coding[J]. IEEE Signal Processing Letters, 2015,22(7): 896-900.
[14]	GAO W , KWONG S , ZHOU Y ,et al. SSIM-based game theory approach for rate-distortion optimized intra frame CTU-level bit allocation[J]. IEEE Transactions on Multimedia, 2016,18(6): 988-999.
[15]	ZHAO H L , XIE W , ZHANG Y C ,et al. An SSIM-motivated LCU-level rate control algorithm for HEVC[C]// 2013 Picture Coding Symposium. Piscataway:IEEE Press, 2013: 85-88.
[16]	LIU X Y , ZHANG Y , ZHU L W ,et al. Perception-based CTU level bit allocation for intra high efficiency video coding[J]. IEEE Access, 2019,7: 154959-154970.
[17]	ZENG H Q , YANG A S , NGAN K N ,et al. Perceptual sensitivity-based rate control method for high efficiency video coding[J]. Multimedia Tools and Applications, 2016,75(17): 10383-10396.
[18]	ZHOU M L , WEI X K , KWONG S ,et al. Just noticeable distortion-based perceptual rate control in HEVC[J]. IEEE Transactions on Image Processing, 2020,29(6): 7603-7614.
[19]	CHOU C H , LI Y C . A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile[J]. IEEE Transactions on Circuits and Systems for Video Technology, 1995,5(6): 467-476.
[20]	WU H R , REIBMAN A R , LIN W S ,et al. Perceptual visual signal compression and transmission[J]. Proceedings of the IEEE, 2013,101(9): 2025-2043.
[21]	YANG K F , WAN S , WU H R ,et al. Detection and estimation of supra-threshold distortion levels of pictures based on just-noticeable difference[C]// 2016 IEEE International Conference on Visual Communications and Image Processing. Piscataway:IEEE Press, 2016: 1-4.
[22]	WU H R , LIN W S , NGAN K N . Rate-perceptual-distortion optimization (RpDO) based picture coding-issues and challenges[C]// 2014 International Conference on Digital Signal Processing. Piscataway:IEEE Press, 2014: 777-782.
[23]	WANG H Q , KATSAVOUNIDIS I , ZHOU J T ,et al. VideoSet:a large-scale compressed video quality dataset based on JND measurement[J]. Journal of Visual Communication and Image Representation, 2017,46(7): 292-302.
[24]	WU J J , LI L D , DONG W S ,et al. Enhanced just noticeable difference model for images with pattern complexity[J]. IEEE Transactions on Image Processing, 2017,26(6): 2682-2693.
[25]	WANG S Q , MA L , FANG Y M ,et al. Just noticeable difference estimation for screen content images[J]. IEEE Transactions on Image Processing, 2016,25(8): 3838-3851.
[26]	张尧尧 . 基于视频结构化的侦查应用[J]. 法制与社会, 2020,19: 75-76.
	ZHAGN Y Y . Investigation application based on video structure[J]. Legal System and Society, 2020,19: 75-76.
[27]	孙展明, 尹伟中 . 论视频侦查的几个基本问题[J]. 公安教育, 2015,12: 32-36.
	SUN Z M , YIN W Z . On several basic issues of video investigation[J]. Public Security Education, 2015,12: 32-36.
[28]	ITU-R. ITU-R BT.500-13-2012:Methodology for the subjective assessment of the quality of television pictures[S]. 2012.
[29]	GONG Y C , WAN S , YANG K F ,et al. Rate-distortion- optimization-based quantization parameter cascading technique for random-access configuration in H.265/HEVC[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017,27(6): 1304-1312.
[30]	LEE J S , EBRAHIMI T . Perceptual video compression:a survey[J]. IEEE Journal of Selected Topics in Signal Processing, 2012,6(6): 684-697.
[31]	YANG X K , LIN W S , LU Z K ,et al. Motion-compensated residue preprocessing in video coding based on just-noticeable-distortion profile[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2005,15(6): 742-752.
[32]	YAMADA K E . Some structural features of the fovea centralis in the human retina[J]. Arch Ophthalmol, 1969,82(2): 151-159.
[33]	ONG P E , YANG X K , LIN W S ,et al. Perceptual quality and objective quality measurements of compressed videos[J]. Journal of Visual Communication and Image Representation, 2006,17(4): 717-737.
[34]	LI B , ZHANG D , LI H Q ,et al. QP determination by lambda value JCTVC-I0426[C]// Proceedings of the 9th Joint Collaborative Team on Video Coding Meeting. New York:ACM Press, 2012: 1-10.
[35]	BOSSEN F , . Common test conditions and software reference configurations[C]// Proceedings of the 11th Joint Collaborative Team on Video Coding Meeting. New York:ACM Press, 2012: 1-4.
[36]	李维, 杨付正, 任鹏 . 考虑视频内容的 H.265/HEVC 帧层码率分配算法[J]. 通信学报, 2015,36(9): 76-81.
	LI W , YANG F Z , REN P . H.265/HEVC frame layer rate allocation algorithm considering video content[J]. Journal on Communications, 2015,36(9): 76-81.
[37]	PATEUXS B , JUNG J . An excel add-in for computing Bj?ntegaard metric and its evolution:VCEG-AE07[C]// Proceedings of the Video Coding Experts Group Meeting. New York:ACM Press, 2007: 1-10.
[38]	DABEER O , CHAUDHURI S . Analysis of an adaptive sampler based on weber's law[J]. IEEE Transactions on Signal Processing, 2011,59(4): 1868-1878.

序列	HM16.0算法	文献[3]算法	文献[4]算法	文献[13]算法	所提算法
BasketballDrill	0.008 5%	0.008 3%	0.001 4%	0.005 6%	0.005 6%
BQMall	0.001 5%	0.001 7%	0.000 3%	0.002 4%	0.001 2%
ParkScene	0.000 7%	0.000 4%	0.001 1%	0.000 7%	0.000 5%
Johnny	0.007 3%	0.004 3%	0.001 4%	0.005 1%	0.000 5%
KristenAndSara	0.002 6%	0.002 1%	0.001 2%	0.005 5%	0.008 3%
Vidyo1	0.004 3%	0.007 8%	0.000 6%	0.001 2%	0.000 7%
Vidyo4	0.000 5%	0.003 6%	0.001 5%	0.004 2%	0.000 4%
Intersection	0.008 7%	0.004 8%	0.001 2%	0.002 3%	0.001 1%
Mainroad	0.014 6%	0.002 1%	0.001 1%	0.001 0%	0.000 9%
Metro	0.012 4%	0.012 1%	0.009 2%	0.000 5%	0.002 6%
Runners	0.000 7%	0.000 7%	0.001 1%	0.000 7%	0.002 0%
Corridor1	0.005 4%	0.015 3%	0.005 0%	0.002 0%	0.001 0%
Corridor2	0.009 9%	0.007 8%	0.002 2%	0.003 9%	0.001 6%
平均	0.005 9%	0.005 5%	0.002 1%	0.002 7%	0.002 0%

序列	文献[3]算法	文献[4]算法	文献[13]算法	所提算法
BasketballDrill	-5.831 3%	2.228 2%	-10.398 5%	-14.437 5%
BQMall	0.989 3%	5.729 4%	-4.267 4%	-11.859 2%
ParkScene	-2.702 7%	-1.367 6%	-8.749 2%	-12.059 7%
Johnny	-6.088 6%	-3.306 6%	-9.238 8%	-17.364 4%
KristenAndSara	-2.578 4%	-3.818 0%	-5.401 1%	-23.322 2%
Vidyo1	-4.817 5%	-2.899 6%	-6.579 8%	-15.795 0%
Vidyo4	-5.450 9%	-3.165 3%	-6.311 9%	-17.130 8%
Intersection	-3.470 8%	-1.563 6%	3.580 3%	-20.574 6%
Mainroad	-3.470 8%	0.986 0%	-10.076 4%	-17.781 3%
Metro	-4.523 0%	3.296 0%	-8.782 5%	-12.545 1%
Runners	-4.472 1%	5.956 9%	-9.845 1%	-16.792 3%
Corridor1	-3.405 8%	-2.162 8%	-7.176 5%	-13.281 8%
Corridor2	-3.119 4%	-3.547 4%	-8.342 5%	-16.792 3%
平均	-3.764 8%	-0.279 6%	-7.045 3%	-16.133 6%

序列	文献[3]算法	文献[4]算法	文献[13]算法	所提算法
BasketballDrill	1.247%	0.830%	0.312%	0.480%
BQMall	-1.214%	-0.699%	-3.337%	-0.045%
ParkScene	-2.600%	0.634%	-1.106%	-0.134%
Johnny	-4.977%	0.014%	0.925%	1.110%
KristenAndSara	-4.578%	-0.466%	0.544%	0.552%
Vidyo1	-5.992%	-0.006%	0.813%	-1.033%
Vidyo4	-4.636%	-0.496%	1.375%	-0.937%
Intersection	-6.964%	0.384%	0.917%	-1.347%
Mainroad	-7.163%	0.320%	-0.405%	0.088%
Metro	-7.101%	-0.826%	0.271%	0.363%
Runners	-6.252%	1.072%	-0.266%	-0.938%
Corridor1	-4.723%	-0.796%	-0.255%	0.215%
Corridor2	-7.150%	0.092%	0.922%	2.641%
平均	-4.777%	0.004%	0.055%	0.078%

视频主观观测实验启发的HEVC感知帧内码率控制

HEVC perceptual intra-frame rate control inspired by video subjective observation experiment

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 38

相关文章 7

Metrics

推荐阅读 0

[1]	郭红伟, 朱策, 杨栩, 罗雷. 基于失真反向传播的时域依赖率失真优化[J]. 通信学报, 2022, 43(12): 222-232.
[2]	李跃,杨高波,丁湘陵,朱亚培. 基于学习模型的3D-HEVC提前Merge模式终止算法[J]. 通信学报, 2019, 40(7): 104-113.
[3]	熊丽荣,雷静之,金鑫. 基于Q-learning的HTTP自适应流码率控制方法研究[J]. 通信学报, 2017, 38(9): 18-24.
[4]	李动节,朱仲杰,王玉儿. 结合视觉感知的对象基嵌入式图像编码中的码率控制方法[J]. 通信学报, 2012, 33(4): 115-120.
[5]	郑新资,骆冰清,孙知信. H.264中一种基于R-Q模型的自适应码率控制算法研究[J]. 通信学报, 2012, 33(12): 35-42.
[6]	朱仲杰,梁丰,蒋刚毅,郁梅. 面向立体视频的比特分配与码率控制算法[J]. 通信学报, 2007, 28(7): 15-21.
[7]	陈云鹏,张培仁,高修峰. 无线视频的一种码率控制方法[J]. 通信学报, 2006, 27(3): 94-98.