基于视频时域感知特性的恰可察觉失真模型

doi:10.11959/j.issn.1000-0801.2022030

Abstract

Abstract:

The existing temporal domain JND(just noticeable distortion) models are not sufficient to depict the interaction between temporal parameters and HVS characteristics, leading to insufficient accuracy of the spatial-temporal JND model.To solve this problem, feature parameters that can accurately describe the temporal characteristics of the video were explored and extracted, as well as a homogenization method for fusing heterogeneous feature parameters, and the temporal domain JND model based on this was improved.The feature parameters were investigated including foreground and background motion, temporal duration along the motion trajectory, residual fluctuation intensity along motion trajectory and adjacent inter-frame prediction residual, etc., which were used to characterize the temporal characteristics.Probability density functions for these feature parameters in the perception sense according to the HVS(human visual system) characteristics were proposed, and uniformly mapping the heterogeneous feature parameters to the scales of self-information and information entropy to achieve a homogeneous fusion measurement.The coupling method of visual attention and masking was explored from the perspective of energy distribution, and the temporal-domain JND weight model was constructed accordingly.On the basis of the spatial JND threshold, the temporal domain weights was integrated to develop a more accurate spatial-temporal JND model.In order to evaluate the performance of the spatiotemporal JND model, a subjective quality evaluation experiment was conducted.Experimental results justify the effectiveness of the proposed model.

Key words: JND, HVS characteristics, visual masking, visual attention, self-information, information entropy

CLC Number:

TN919

Yafen1 XING, Haibing YIN, Hongkui WANG, Qionghua LUO. Video temporal perception characteristics based just noticeable difference model[J]. Telecommunications Science, 2022, 38(2): 92-102.

Figures/Tables 8

References 25

[1]	YUAN D , ZHAO T S , XU Y W ,et al. Visual JND:a perceptual measurement in video coding[J]. IEEE Access, 2019(7): 29014-29022.
[2]	KORHONEN J . Two-level approach for no-reference consumer video quality assessment[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2019,28(12): 5923-5938.
[3]	CHOU C H , LI Y C . A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile[J]. IEEE Transactions on Circuits and Systems for Video Technology, 1995,5(6): 467-476.
[4]	YANG X K , LING W S , LU Z K ,et al. Just noticeable distortion model and its applications in video coding[J]. Signal Processing:Image Communication, 2005,20(7): 662-680.
[5]	CHIN Y J , BERGER T . A software-only videocodec using pixelwise conditional differential replenishment and perceptual enhancements[J]. IEEE Transactions on Circuits and Systems for Video Technology, 1999,9(3): 438-450.
[6]	KELLY D H . Motion and vision.II.Stabilized spatio-temporal threshold surface[J]. Journal of the Optical Society of America, 1979,69(10): 1340-1349.
[7]	DALY S . Engineering observations from spatiovelocity and spatiotemporal visual models[M]. Vision Models and Applications to Image and Video Processing. Heidelberg: Springer, 2001: 179-200.
[8]	JIA Y , LIN W , KASSIM A A . Estimating just-noticeable distortion for video[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2006,16(7): 820-829.
[9]	WEI Z Y , NGAN K N . Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2009,19(3): 337-346.
[10]	BAE S H , KIM M . A DCT-based total JND profile for spatiotemporal and foveated masking effects[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017,27(6): 1196-1207.
[11]	WANG H Q , GAN W H , HU S D ,et al. MCL-JCV:a JND-based H.264/AVC video quality assessment dataset[C]// Proceedings of 2016 IEEE International Conference on Image Processing. Piscataway:IEEE Press, 2016: 1509-1513.
[12]	WANG H Q , KATSAVOUNIDIS I , ZHOU J T ,et al. VideoSet:a large-scale compressed video quality dataset based on JND measurement[J]. Journal of Visual Communication and Image Representation, 2017,46: 292-302.
[13]	KI S , BAE S H , KIM M ,et al. Learning-based just-noticeable-quantization- distortion modeling for perceptual video coding[J]. IEEE Transactions on Image Processing, 2018,27(7): 3178-3193.
[14]	LIU H H , ZHANG Y , ZHANG H ,et al. Deep learning-based picture-wise just noticeable distortion prediction model for image compression[J]. IEEE Transactions on Image Processing, 2020,29: 641-656.
[15]	ZHANG Y , LIU H H , YANG Y ,et al. Deep learning based just noticeable difference and perceptual quality prediction models for compressed video[J]. IEEE Transactions on Circuits and Systems for Video Technology, 6224(99): 1.
[16]	WANG Z , LI Q . Video quality assessment using a statistical model of human visual speed perception[J]. Journal of the Optical Society of America A,Optics,Image Science,and Vision, 2007,24(12): B61-B69.
[17]	MACKNIK S L , LIVINGSTONE M S . Neuronal correlates of visibility and invisibility in the primate visual system[J]. Nature Neuroscience, 1998,1(2): 144-149.
[18]	STOCKER A A , SIMONCELLI E P . Noise characteristics and prior expectations in human visual speed perception[J]. Nature Neuroscience, 2006,9(4): 578-585.
[19]	PETERSON H A , AHUMADA A J J , WATSON A B . Improved detection model for DCT coefficient quantization[C]// Proceedings of the Human Vision,Visual Processing,and Digital Dis play IV,F,1993.[S.l.:s.n.], 1993: 191-201.
[20]	BARKOWSKY M , BIALKOWSKI J , ESKOFIER B ,et al. Temporal trajectory aware video quality measure[J]. IEEE Journal of Selected Topics in Signal Processing, 2009,3(2): 266-279.
[21]	HU S D , JIN L N , WANG H L ,et al. Objective video quality assessment based on perceptually weighted mean squared error[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017,27(9): 1844-1855.
[22]	CHEN Z , GUILLEMOT C . Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2010,20(6): 806-819.
[23]	WU J , QI F , SHI G ,et al. Non-local spatial redundancy reduction for bottom-up saliency estimation[J]. Journal of Visual Communication and Image Representation, 2012,23(7): 1158-1166.
[24]	崔帅南, 彭宗举, 邹文辉 ,等. 多特征融合的合成视点立体图像质量评价[J]. 电信科学, 2019,35(5): 104-112.
	CUI S N , PENG Z J , ZOU W H ,et al. Quality assessment of synthetic viewpoint stereo image with multi-feature fusion[J]. Telecommunications Science, 2019,35(5): 104-112.
[25]	ZENG Z P , ZENG H Q , CHEN J ,et al. Visual attention guide dpixel-wise just noticeable difference model[J]. IEEE Access, 2019,7: 132111-132119.

Metrics

Recommended 0

No Suggested Reading articles found!

测试序列	Wei^[9]提出的模型			Bae^[10]提出的模型			Zeng^[25]提出的模型			本文所提模型
测试序列	PSNR	VMAF	DMOS	PSNR	VMAF	DMOS	PSNR	VMAF	DMOS	PSNR	VMAF	DMOS
Ba	26.66	99.90	12.67	27.09	99.76	15.91	33.53	99.80	19.52	25.88	99.98	9.33
BT	26.48	99.17	16.50	25.76	98.26	19.65	34.34	99.61	18.39	25.40	99.31	12.89
CT	28.20	97.36	11.95	28.73	96.88	11.67	31.33	94.35	22.58	27.89	98.78	7.38
Ki	27.50	98.20	15.58	27.55	95.49	22.20	29.92	94.27	22.67	27.11	98.90	13.32
Pa	28.21	97.56	16.89	28.20	95.37	18.11	30.41	93.52	20.12	27.69	98.26	15.11
BD	28.80	99.76	14.33	29.53	99.33	12.83	33.61	98.59	19.12	28.17	99.95	11.50
BM	28.16	99.76	12.67	30.00	99.84	10.78	32.09	99.15	17.33	26.43	99.97	9.33
PS	28.26	96.72	13.33	28.72	96.08	15.00	32.62	95.14	19.65	27.91	97.94	11.20
RH	26.88	99.98	15.17	27.88	99.90	18.24	32.56	99.96	17.17	25.89	99.98	14.89
平均值	27.68	98.71	14.34	28.16	97.88	16.04	32.27	97.16	19.62	26.93	99.23	11.66

Video temporal perception characteristics based just noticeable difference model

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 25

Related Articles 5

Metrics

Recommended 0

[1]	Qionghua LUO, Hongkui WANG, Haibing YIN, Yafen XING. Just noticeable distortion model based on entropy masking in DCT domain [J]. Telecommunications Science, 2023, 39(2): 59-70.
[2]	Panpan LI, Zhengxia XIE, Zengkai WANG, Rui JIN. Calculation method of information dissemination based on information entropy in public internet [J]. Telecommunications Science, 2022, 38(4): 90-100.
[3]	Jinhua WANG,Na YING,Chendu ZHU,Zhaosen LIU,Zhedong CAI. Speech emotion recognition algorithm based on spectrogram feature extraction of deep space attention feature [J]. Telecommunications Science, 2019, 35(7): 100-108.
[4]	Minjuan WANG,Zhengpeng JI,Chao LV. A rules extraction algorithm for IPTV customers forecasting based on the forecasting entropy measurement [J]. Telecommunications Science, 2016, 32(5): 160-165.
[5]	Shaomin Zhu,Zhiqiang Zhang. Application of Hybrid Transform Domain Digital Watermarking in Power System Information Security [J]. Telecommunications Science, 2013, 29(11): 82-86.