视频帧率上转换检测技术综述

doi:10.11959/j.issn.2096-109x.2018085

摘要/Abstract

摘要：

视频帧率上转换检测技术是视频取证技术的一种。为了系统阐述视频上转换检测领域现状，合理导向后续研究，对相关技术进行了综述。首先对相关研究历史和发展进程进行阐述，总结上转换概念及技术框架。然后根据检测技术的目的，对现有算法分类阐述。最后汇总介绍了视频帧率上转换检测领域的主要研究团队及其研究成果。从算法框架、检测结果等方面对比现有检测技术，提出了两点展望。视频帧率上转换作为视频后处理技术的重要组成部分，目前仍需进一步研究。

关键词: 视频篡改取证, 帧率上转换, 帧率上转换检测, 周期性分析, 篡改算法分类识别

Abstract:

Video frame rate up-conversion detection technology is a kind of video forensics technology.In order to systematically elaborate the research progress in the field of video up-conversion detection and guide the follow-up research of video up-conversion detection algorithms,the main algorithms of up-conversion detection technology are summarized.First of all,elaborate the relevant research history and development process and summarize the up-conversion concept and technical framework.According to the purpose of the detection technology,classify and explain the existing algorithms.Finally,the main research teams in the field and their research results were summarized.From the aspects of algorithm framework and test results,compare the existing detection techniques and propose two prospects.Video frame rate up-conversion is an important part of video post-processing technology,and still need further research.

Key words: video forensics, frame rate up conversion, frame rate up conversion detection, periodic analysis, tamper algorithm identification and classification

中图分类号:

TP391

何天琦,蒋兴浩,孙锬锋. 视频帧率上转换检测技术综述[J]. 网络与信息安全学报, 2018, 4(10): 1-11.

Tianqi HE,Xinghao JIANG,Tanfeng SUN. Review of video frame rate up conversion detection[J]. Chinese Journal of Network and Information Security, 2018, 4(10): 1-11.

图/表 12

表1

图1

图2

表2

图3

图4

图5

图6

表3

表4

表5

表6

参考文献 44

[1]	GIRONI A , FONTANI M , BIANCHI T . A video forensic technique for detecting frame deletion and insertion[J]. IEEE International Conference on Acoustics,Speech and Signal Processing, 2014: 6226-6230.
[2]	LUO W , QU Z , PAN F ,et al. A survey of passive technology for digital image forensics[J]. Frontiers of Computer Science in China, 2008,1(2): 166-179.
[3]	黄昆仑, 白蔚 . 视频帧率上转换技术综述[J]. 数字通信世界, 2011(5): 82-84.
	HUANG K L , BAI W . Overview of video frame rate up-conversion technology[J]. Digital communication world, 2011(5): 82-84.
[4]	LUO W , WU M , HUANG J . MPEG recompression detection based on block artifacts[J]. Proceedings of SPIE-The International Society for Optical, 2008,6819:68190X-68190X-12.
[5]	DING X , YANG G , Li R . Identification of motion-compensated frame rate up-conversion based on residual signal[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2017,PP(99): 1-1.
[6]	BESTAGINI P , BATTAGLIA S , MILANI S ,et al. Detection of temporal interpolation in video sequences[C]// IEEE International Conference on Acoustics,Speech and Signal Processing. 2013: 3033-3037.
[7]	BIAN S , LUO W , HUANG J . Detecting video frame-rate up-conversion based on periodic properties of inter-frame similarity[J]. Multimedia Tools ＆ Applications, 2014,72(1): 437-451.
[8]	夏明, 杨高波, 黄勤珍 . 使用PSNR周期特性检测视频帧率上转伪造[J]. 计算机应用研究, 2016,33(8): 2409-2412.
	XIA M , YANG G B , HUANG Q Z . Detecting frame-rate up-conversion forgery in videos using periodicity of PSNR change[J]. Application Research of Computers, 2016,33(8): 2409-2412.
[9]	YAO Y , YANG G , SUN X . Detecting video frame-rate up-conversion based on periodic properties of edge-intensity[J]. Journal of Information Security ＆ Applications, 2016,26(C): 39-50.
[10]	XIA M , YANG G , LI L . Detecting video frame rate up-conversion based on frame-level analysis of average texture variation[J]. Multimedia Tools ＆ Applications, 2017,76(6): 8399-8421.
[11]	JUNG D J , LEE H K . Frame-rate conversion detection based on periodicity of motion artifact[J]. Multimedia Tools ＆ Applications, 2017(1): 1-22.
[12]	CHAO J , JIANG X , SUN T . A novel video inter-frame forgery model detection scheme based on optical flow consistency[M]. Digital Forensics and Watermarking. Springer Berlin Heidelberg, 2013: 267-281.
[13]	WU Y , JIANG X , SUN T . Exposing video inter-frame forgery based on velocity field consistency[C]// IEEE International Conference on Acoustics,Speech and Signal Processing. IEEE, 2014: 2674-2678.
[14]	LIU Y , HUANG T . Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis[J]. Multimedia Systems, 2017,23(2): 223-238.
[15]	ORCHARD M T , SULLIVAN G J.Overlapped block motion compensation:an estimation-theoretic approach . [J]. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 1994,3(5): 693-9.
[16]	KANG S J , CHO K R , KIM Y H . Motion compensated frame rate up-conversion using extended bilateral motion estimation[J]. IEEE Transactions on Consumer Electronics, 2008,53(4): 1759-1767.
[17]	CHOI B D , HAN J W , KIM C S . Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2007,17(4): 407-416.
[18]	KANG S J , YOO S , KIM Y H . Dual motion estimation for frame rate up-conversion[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2011,20(12): 1909-1914.
[19]	LI R , GAN Z , CUI Z . Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual-weighted overlapped block motion compensation[J]. Journal of Display Technology, 2014,10(12): 1010-1023.
[20]	YOO D G , KANG S J , KIM Y H . Direction-select motion estimation for motion-compensated frame rate up-conversion[J]. IEEE/OSA Journal of Display Technology, 2013,9(10): 840-850.
[21]	LIN H Y , TSAI T H . High visual quality particle based frame rate up conversion with acceleration assisted motion trajectory calibration[J]. Journal of Display Technology, 2012,8(6): 341-351.
[22]	LIU H , XIONG R , ZHAO D . Multiple hypotheses bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2012,22(8): 1188-1198.
[23]	陈伟 . 基于运动估计和运动补偿的帧率上转换算法研究[D]. 哈尔滨工业大学, 2016.
	CHEN W . Research on frame rate up-conversion algorithm based on motion estimation and motion compensation[D]. Harbin Institute of Technology, 2016.
[24]	KANG S J , CHO K R , KIM Y H . Motion compensated frame rate up-conversion using extended bilateral motion estimation[J]. IEEE Transactions on Consumer Electronics, 2008,53(4): 1759-1767.
[25]	ORCHARD M T , SULLIVAN G J . Overlapped block motion compensation:an estimation-theoretic approach[J]. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 1994,3(5): 693-9.
[26]	CHOI B D , HAN J W , Kim C S ,et al. Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2007,17(4): 407-416.
[27]	KANG S J , YOO S , KIM Y H . Dual motion estimation for frame rate up-conversion[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2011,20(12): 1909-1914.
[28]	胡圆圆 . 基于视觉显著性的视频关键帧提取与帧速率上转换[D]. 南京邮电大学, 2016.
	HU Y Y . Video key frame extraction and frame rate up conversion based on vision saliency[D]. Nanjing University of Posts and Telecommunications, 2016.
[29]	LI R , GAN Z , CUI Z . Multi-channel mixed-pattern based frame rate up-conversion using spatio-temporal motion vector refinement and dual-weighted overlapped block motion compensation[J]. Journal of Display Technology, 2014,10(12): 1010-1023.
[30]	YOO D G , KANG S J , KIM Y H . Direction-select motion estimation for motion-compensated frame rate up-conversion[J]. IEEE/OSA Journal of Display Technology, 2013,9(10): 840-850.
[31]	MAHDIAN B , SAIC S . Blind authentication using periodic properties of interpolation[J]. IEEE Transactions on Information Forensics＆ Security, 2008,3(3): 529-538.
[32]	DIETTERICH T G , BAKIRI G . Solving multiclass learning problems via error correcting output codes[J]. Journal of Artificial Intelligence Research, 1995,2(1): 263-286.
[33]	POPESCU A C , FARID H . Exposing digital forgeries by detecting traces of resampling[J]. IEEE Transactions on Signal Processing, 2005,53(2): 758-767.
[34]	VAZQUEZ-PADIN D , COMESANA P . ML estimation of the resampling factor[C]// IEEE International Workshop on Information Forensics and Security. IEEE, 2012: 205-210.
[35]	KIRCHNER M , . Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue[C]// The Workshop on Multimedia ＆ Security. ACM, 2008: 11-20.
[36]	HUANG H C , FANG W C . Metadata-based image watermarking for copyright protection[J]. Simulation Modelling Practice ＆ Theory, 2010,18(4): 436-445.
[37]	KOBAYASHI M , OKABE T , SATO Y . Detecting forgery from static-scene video based on inconsistency in noise level functions[J]. IEEE Transactions on Information Forensics ＆ Security, 2010,5(4): 883-892.
[38]	LIAO D , YANG R , LIU H.Double H . 264/AVC compression detection using quantized nonzero AC coefficients[J]. Proceedings of SPIE - The International Society for Optical Engineering, 2011,7880:78800Q-78800Q-10.
[39]	TEKALP A M . Digital video processing[M]. Englewood Cliffs,NJ: Prentice Hall, 1995.
[40]	KAUFMAN P J . Smarter Trading:Improving Performance in Changing Markets[M]. McGraw-Hill; 1995:p 138-51.
[41]	KIM U S , SUNWOO M H . New frame rate up-conversion algorithms with low computational complexity[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2014,24(3): 384-393.
[42]	TSAI T H , Lin H Y . High visual quality particle based frame rate up conversion with acceleration assisted motion trajectory calibration[J]. Journal of Display Technology, 2012,8(6): 341-351.
[43]	LIU H , XIONG R , Zhao D . Multiple hypotheses Bayesian frame rate up-conversion by adaptive fusion of motion-compensated interpolations[J]. IEEE Transactions on Circuits ＆ Systems for Video Technology, 2012,22(8): 1188-1198.
[44]	DAR Y , BRUCKSTEIN A M . Motion-compensated coding and frame rate up-conversion:models and analysis[J]. IEEE Transactions on Image Processing, 2015,24(7): 2051-2066.

年份	团队	成果	场景
2013年	米兰理工大学电子与信息学院Paolo Bestagini^[6]团队	首次提出上转换痕迹检测^[6]	短片段视频、数帧数字图像集
2013年	中山大学黄继武^[7]团队	首次明确对连续视频进行检测^[7]	特定软件获得的上转换视频
2015—2018年	湖南大学杨高波^[5,8-10]团队、南京信息工程大学孙星明^[8]团队、中国矿业大学李雷达^[10]团队	采用更加广泛且典型的帧率篡改手段，获得更为置信的实验样本；提取各类痕迹特征	适用于推断原始帧率、定位篡改点的环境
2017年	韩国国防科学研究所 Dae-Jin Jung 团队联合KAIST的Heung Kyu Lee^[11]团队	考虑到上转换操作对连续视频中目标和背景的运动，提出了新的痕迹特征	适用于包含目标运动场景的连续视频

上转换篡改算法	篡改检测算法
基于非运动补偿	Bian^[7]等
基于运动补偿	Bestagini^[6]等、Yao^[9]等、Ming Xia^[8,10]等、Dae-Jin Jung^[11]等、Ding^[5]等

算法	样本
基于残差序列分布	3段通用影像传输格式（CIF,common intermediate format）视频：Foreman、Hall、Mobile
基于帧间相似度分布	100段未压缩视频及400段H.264/AVC压缩编码格式视频；分辨率从176×144至1 920×1 080
基于边缘密度分布	5段未压缩视频，包含4类：单一目标强运动、单一目标弱运动、多目标强运动、多目标弱运动。分辨率
	从176×144至1 920×1 080
基于纹理变化分布	30段未压缩CIF视频，30段H.264/ AVC格式视频
基于运动效应分布	50段CIF视频，分辨率从176×144至1 920×1 080不等，每段序列不超过300帧；包含5种压缩参数（100，
	90，80，70，60）的H.264/AVC格式视频
基于差值信息分布	15段CIF视频和10段高清视频；对应采用5种不同图像组（GOP，group of picture）；量化参数配置获得
	MPEG-2和H.264/AVC格式视频

算法	平均准确率
基于残差序列分布	1) 上转换成25帧/秒，45帧/秒，60帧/秒，90帧/秒：视频达到50帧，准确率突变到100%；
	2) 上转换成120帧/秒：视频达到50帧，准确率激增至70%，随后在达到600帧时，上升到100%；
	3) 上转换成36帧/秒：视频帧数在200帧内，准确率逐渐上升，直至100%
基于帧间相似度分布	1) 未压缩视频：大于99%；
	2) H.264/AVC压缩视频：大于97%
基于边缘密度分布	1) 未压缩视频：94.05%；
	2) H.264/AVC压缩视频：93.2%
基于平均纹理变化分布	1) OBMC篡改方式：最高为95.2%；
	2) 帧平均篡改方式：最高为93.5%；
	3) 采用软件MSU、MVTools：准确率分别为94.5%、94.4%
基于运动效应分布	1) 帧复制方式：97%；
	2) 帧平均方式：78%；
	3) 运动补偿方式：90%
基于差值信息分布	1) 未压缩视频：97%；
	2) H.264/AVC压缩视频：95.16%；
	3) MPEG-2压缩视频：83.24%；
	4) 分类识别篡改方式：81.27%

算法	优点	缺点
基于残差序列的算法	适用于小段视频检测	1) 对帧复制、帧平均的篡改视频检测准确率不高，易误判；
		2) 算法检测准确率与视频序列长度有较强相关性，视频帧数增加时，检测复杂度线性增加
基于帧间相似度的算法	样本全面，置信度高	仅用ImTOO^[7]单一转换软件，且该软件仅采用帧复制方式
基于边缘密度的算法	可检测帧复制、帧平均、运动补偿方式的篡改视频	边缘检测算子选择困难
基于平均纹理变化的算法	1) 从原始帧序列中自动区分上转换视频序列；	当待检测视频序列包含快速场景切换时，帧平均纹理差值的异常会导致插值帧误判为原始帧，降低了检测准确率
	2) 低时间复杂度
基于运动效应的算法	考虑了帧率下转换的情况，并对应进行了分析	当帧率24帧/秒上转换至25帧/秒时，该方法检测准确率急剧降低