HRDA-Net：面向真实场景的图像多篡改检测与定位算法

doi:10.11959/j.issn.1000-436x.2022016

Abstract

Abstract:

Aiming at the problems that the fake image just contains one tampered operation in mainstream manipulation datasets and the artifact is a common problem in manipulation location.The multiple manipulation dataset (MM Dataset) was constructed for real scene, which contained both splicing and removal in each images.Based on this, an end-to-end high-resolution representation dilation attention network (HRDA-Net) was proposed for multiple manipulation detection and localization, which fused the RGB and SRM features through the top-down dilation convolutional attention (TDDCA).Finally, the mixed dilated convolution (MDC) would respectively extract the features of splicing and removal, which could realize multiple manipulation location and confidence prediction.The cosine similarity loss was proposed as auxiliary loss to improve the efficiency of network.Experimental results on MM Dataset indicate that the performance and robustness of HRDA-Net is better than semantic segmentation methods.Furthermore, the scores of F1 and AUC are greater than state-of-the-art manipulation location methods in CASIA and NIST datasets.

Key words: deep learning, multiple manipulation detection and location, MM Dataset, cosine similarity loss function

CLC Number:

TP391

Ye ZHU, Yilin YU, Yingchun GUO. HRDA-Net: image multiple manipulation detection and location algorithm in real scene[J]. Journal on Communications, 2022, 43(1): 217-226.

Figures/Tables 13

HRNet	SRM DB	TDDCA	MDC	L_cos-定位	L_cos-检测	拼接-F1	移除-F1	fp	Accuracy
√	×	×	×	√	×	0.804	0.485	0.303	0.766
√	√	×	×	√	×	0.490	0.156	0.365	0.692
√	√	√	×	√	×	0.894	0.558	0.162	0.828
√	√	×	√	√	×	0.745	0.413	0.177	0.775
√	√	√	√	×	×	0.891	0.564	0.152	0.836
√	√	√	√	√	√	0.899	0.576	0.150	0.833
√	√	√	√	√	×	$0 . 899$	$0 . 576$	$0 . 150$	$0 . 850$

训练方法	拼接-F1	移除-F1	fp	Accuracy
多篡改定位	0.899	0.576	0.150	—
多篡改检测	—	—	—	0.892
分步训练-参数不冻结	0.537	0.221	0.246	$0 . 913$
分步训练-参数冻结	$0 . 899$	$0 . 576$	$0 . 150$	0.850

模型名称	拼接篡改定位			移除篡改定位
模型名称	precise	recall	F1	precise	recall	F1
FCN	$0 . 962$	0.616	0.636	0.580	0.225	0.305
Deeplabv3	0.831	0.727	0.770	0.760	0.385	0.507
PSPNet	0.808	0.685	0.734	0.581	0.327	0.407
DANet	0.718	0.799	0.751	0.717	0.228	0.344
RRU-Net	0.690	0.793	0.727	0.457	0.224	0.286
HRNet	0.867	0.768	0.804	0.700	0.388	0.485
HRDA-Net	0.888	$0 . 913$	$0 . 899$	$0 . 764$	$0 . 481$	$0 . 576$

模型	F1		AUC
模型	NIST	CASIA	NIST	CASIA
NoI	0.285	0.263	0.487	0.612
CFA	0.174	0.207	0.501	0.522
RGB-N	0.722	0.408	0.937	0.795
LSTM-En	—	0.391	0.793	0.762
GSCNet	0.837	0.471	0.917	$0 . 833$
SEINet	0.891	0.488	0.980	0.801
HRDA-Net	$0 . 951$	$0 . 496$	$0 . 993$	$0 . 833$

References 39

[1]	乔通, 姚宏伟, 潘彬民 ,等. 基于深度学习的数字图像取证技术研究进展[J]. 网络与信息安全学报, 2021,7(5): 13-28.
	QIAO T , YAO H W , PAN B M ,et al. Research progress of digital image forensic techniques based on deep learning[J]. Chinese Journal of Network and Information Security, 2021,7(5): 13-28.
[2]	田秀霞, 李华强, 张琴 ,等. 基于双通道R-FCN的图像篡改检测模型[J]. 计算机学报, 2021,44(2): 370-383.
	TIAN X X , LI H Q , ZHANG Q ,et al. Dual-channel R-FCN model for image forgery detection[J]. Chinese Journal of Computers, 2021,44(2): 370-383.
[3]	张旭, 胡晰远, 陈晨 ,等. 基于透视投影下空间光照一致性分析的图像拼接篡改检测[J]. 自动化学报, 2019,45(10): 1857-1869.
	ZHANG X , HU X Y , CHEN C ,et al. Image splicing detection based on spatial lighting consistency analysis under perspective projection[J]. Acta Automatica Sinica, 2019,45(10): 1857-1869.
[4]	RAO Y , NI J Q . A deep learning approach to detection of splicing and copy-move forgeries in images[C]// Proceedings of 2016 IEEE International Workshop on Information Forensics and Security (WIFS). Piscataway:IEEE Press, 2016: 1-6.
[5]	ROTA P , SANGINETO E , CONOTTER V ,et al. Bad teacher or unruly student:can deep learning say something in Image Forensics analysis?[C]// Proceedings of 2016 23rd International Conference on Pattern Recognition (ICPR). Piscataway:IEEE Press, 2016: 2503-2508.
[6]	LIU B , PUN C M . Locating splicing forgery by fully convolutional networks and conditional random field[J]. Signal Processing:Image Communication, 2018,66: 103-112.
[7]	BI X L , WEI Y , XIAO B ,et al. RRU-net:the ringed residual U-net for image splicing forgery detection[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway:IEEE Press, 2019: 1-10.
[8]	王珠珠 . 基于 U 型检测网络的图像篡改检测算法[J]. 通信学报, 2019,40(4): 171-178.
	WANG Z Z . Image forgery detection algorithm based on U-shaped detection network[J]. Journal on Communications, 2019,40(4): 171-178.
[9]	CHEN B J , TAN W J , COATRIEUX G ,et al. A serial image copy-move forgery localization scheme with source/target distinguishment[J]. IEEE Transactions on Multimedia, 2021,23: 3506-3517.
[10]	ZHU Y , CHEN C F , YAN G ,et al. AR-net:adaptive attention and residual refinement network for copy-move forgery detection[J]. IEEE Transactions on Industrial Informatics, 2020,16(10): 6714-6723.
[11]	LI H D , HUANG J W . Localization of deep inpainting using high-pass fully convolutional network[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2019: 8301-8310.
[12]	LIU Y Q , GUAN Q X , ZHAO X F ,et al. Image forgery localization based on multi-scale convolutional neural networks[C]// Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security. New York:ACM Press, 2018: 85-90.
[13]	BAPPY J H , SIMONS C , NATARAJ L ,et al. Hybrid LSTM and encoder-decoder architecture for detection of image forgeries[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2019,28(7): 3286-3300.
[14]	ZHOU P , HAN X T , MORARIU V I ,et al. Learning rich features for image manipulation detection[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 1053-1061.
[15]	WU Y , ABDALMAGEED W , NATARAJAN P . ManTra-net:manipulation tracing network for detection and localization of image forgeries with anomalous features[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 9543-9552.
[16]	HU X F , ZHANG Z H , JIANG Z Y ,et al. SPAN:spatial pyramid attention network for image manipulation localization[C]// Proceedings of the European Conference on Computer Vision (ECCV). Berlin:Springer, 2020: 312-328.
[17]	SUN K , XIAO B , LIU D ,et al. Deep high-resolution representation learning for human pose estimation[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 5693-5703.
[18]	DONG J , WANG W , TAN T N . CASIA image tampering detection evaluation database[C]// Proceedings of 2013 IEEE China Summit and International Conference on Signal and Information Processing. Piscataway:IEEE Press, 2013: 422-426.
[19]	GUAN H Y , KOZAK M , ROBERTSON E ,et al. MFC datasets:large-scale benchmark datasets for media forensic challenge evaluation[C]// Proceedings of 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). Piscataway:IEEE Press, 2019: 63-72.
[20]	WEN B H , ZHU Y , SUBRAMANIAN R ,et al. COVERAGE—a novel database for copy-move forgery detection[C]// Proceedings of 2016 IEEE International Conference on Image Processing (ICIP). Piscataway:IEEE Press, 2016: 161-165.
[21]	MAHFOUDI G , TAJINI B , RETRAINT F ,et al. DEFACTO:image and face manipulation dataset[C]// Proceedings of 2019 27th European Signal Processing Conference (EUSIPCO). Piscataway:IEEE Press, 2019: 1-5.
[22]	LIN T Y , MAIRE M , BELONGIE S ,et al. Microsoft COCO:common objects in context[C]// European Conference on Computer Vision. Berlin:Springer, 2014: 740-755.
[23]	HU J , SHEN L , SUN G . Squeeze-and-excitation networks[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 7132-7141.
[24]	WOO S , PARK J , LEE J Y ,et al. CBAM:convolutional block attention module[C]// European Conference on Computer Vision. Berlin:Springer, 2018: 3-19.
[25]	ZHANG Q L , YANG Y B . SA-net:shuffle attention for deep convolutional neural networks[C]// Proceedings of 2021 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2021: 2235-2239.
[26]	PANG B , LI Y Z , LI J F ,et al. TDAF:top-down attention framework for vision tasks[J]. arXiv Preprint,arXiv:2012.07248, 2020.
[27]	SHORE J , JOHNSON R . Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy[J]. IEEE Transactions on Information Theory, 1980,26(1): 26-37.
[28]	LIN T Y , GOYAL P , GIRSHICK R ,et al. Focal loss for dense object detection[C]// Proceedings of the IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 2980-2988.
[29]	WANG H , WANG Y T , ZHOU Z ,et al. CosFace:large margin cosine loss for deep face recognition[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 5265-5274.
[30]	HW A , CL A , ML A ,et al. Optimized HRNet for image semantic segmentation[J]. Expert Systems with Applications, 2020,174:114532.
[31]	HUANG G , LIU Z , VAN DER MAATEN L ,et al. Densely connected convolutional networks[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 4700-4708.
[32]	LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2015: 3431-3440.
[33]	CHEN L C , PAPANDREOU G , SCHROFF F ,et al. Rethinking atrous convolution for semantic image segmentation[J]. arXiv Preprint,arXiv:1706.05587, 2017.
[34]	ZHAO H S , SHI J P , QI X J ,et al. Pyramid scene parsing network[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 2881-2890.
[35]	FU J , LIU J , TIAN H J ,et al. Dual attention network for scene segmentation[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 3146-3154.
[36]	MAHDIAN B , SAIC S . Using noise inconsistencies for blind image forensics[J]. Image and Vision Computing, 2009,27(10): 1497-1503.
[37]	FERRARA P , BIANCHI T , ROSA A D ,et al. Image forgery localization via fine-grained analysis of CFA artifacts[J]. IEEE Transactions on Information Forensics and Security, 2012,7(5): 1566-1577.
[38]	SHI Z N , SHEN X J , CHEN H P ,et al. Global semantic consistency network for image manipulation detection[J]. IEEE Signal Processing Letters, 2020,27: 1755-1759.
[39]	ZHU Y , QI N , GUO Y ,et al. SEINet:semantic-edge interaction network for image manipulation localization[C]// Proceedings of the fourth Chinese Conference on Pattern Recognition and Computer Vision.[S.l.:s.n.], 2021: 1-13.

Metrics

Recommended 0

No Suggested Reading articles found!

数据集	篡改图像数量/幅	用途
自制拼接数据集	15 000	预训练
MM Dataset	800/200	微调/测试
CASIA v2.0	5 059	微调
CASIA v1.0	908	测试
NIST	404/160	微调/测试

后处理手段	参数名称	参数取值
JPEG压缩	压缩因子	{100,90,80,70,60,50}
高斯噪声	噪声参数	{0.06,0.05,0.04,0.03,0.02,0.01}
高斯模糊	半径	{1.0, 1.2,1.4,1.6,1.8,2.0}
亮度	—	{1.0, 1.1, 1.2, 1.3, 1.4, 1.5}
对比度	—	{1.0, 1.1, 1.2, 1.3, 1.4, 1.5}
色彩平衡	—	{1.0, 1.1, 1.2, 1.3, 1.4, 1.5}

HRDA-Net: image multiple manipulation detection and location algorithm in real scene

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 39

Related Articles 15

Metrics

Recommended 0

[1]	Dongyu CHEN, Hua CHEN, Limin FAN, Yifang FU, Jian WANG. Research on test strategy for randomness based on deep learning [J]. Journal on Communications, 2023, 44(6): 23-33.
[2]	Rongpeng LI, Bingyan WANG, Honggang ZHANG, Zhifeng ZHAO. Design of knowledge enhanced semantic communication receiver [J]. Journal on Communications, 2023, 44(6): 70-76.
[3]	Shuai MA, Ke PEI, Huayan QI, Hang LI, Wen CAO, Hongmei WANG, Hailiang XIONG, Shiyin LI. Research on geomagnetic indoor high-precision positioning algorithm based on generative model [J]. Journal on Communications, 2023, 44(6): 211-222.
[4]	Jie YANG, Biao DONG, Xue FU, Yu WANG, Guan GUI. Lightweight decentralized learning-based automatic modulation classification method [J]. Journal on Communications, 2022, 43(7): 134-142.
[5]	Xiuzhang YANG, Guojun PENG, Zichuan LI, Yangqi LYU, Side LIU, Chenguang LI. Research on entity recognition and alignment of APT attack based on Bert and BiLSTM-CRF [J]. Journal on Communications, 2022, 43(6): 58-70.
[6]	Yong LIAO, Shiyi WANG. CSI feedback algorithm based on RM-Net for massive MIMO systems in high-speed mobile environment [J]. Journal on Communications, 2022, 43(5): 166-176.
[7]	Yurong LIAO, Haining WANG, Cunbao LIN, Yang LI, Yuqiang FANG, Shuyan NI. Research progress of deep learning-based object detection of optical remote sensing image [J]. Journal on Communications, 2022, 43(5): 190-203.
[8]	Zenghua ZHAO, Yuefan TONG, Jiayang CUI. Device-independent Wi-Fi fingerprinting indoor localization model based on domain adaptation [J]. Journal on Communications, 2022, 43(4): 143-153.
[9]	Yong LIAO, Gang CHENG, Yujie LI. CSI feedback algorithm based on deep unfolding for massive MIMO systems [J]. Journal on Communications, 2022, 43(12): 77-88.
[10]	Xueyuan DUAN, Yu FU, Kun WANG, Bin LI. LDoS attack detection method based on simple statistical features [J]. Journal on Communications, 2022, 43(11): 53-64.
[11]	Junyan HUO, Ruipeng QIU, Yanzhuo MA, Fuzheng YANG. Reference frame list optimization algorithm in video coding by quality enhancement of the nearest picture [J]. Journal on Communications, 2022, 43(11): 136-147.
[12]	Haiyan KANG, Yuanrui JI. Research on federated learning approach based on local differential privacy [J]. Journal on Communications, 2022, 43(10): 94-105.
[13]	Hongxia ZHANG, Qi WANG, Dengyue WANG, Ben WANG. Honeypot contract detection of blockchain based on deep learning [J]. Journal on Communications, 2022, 43(1): 194-202.
[14]	Yan YAN, Yiming CONG, Mahmood Adnan, Quanzheng SHENG. Statistics release and privacy protection method of location big data based on deep learning [J]. Journal on Communications, 2022, 43(1): 203-216.
[15]	Xiaoyuan YANG, Xinliang BI, Jia LIU, Siyuan HUANG. High-capacity image steganography algorithm combining image encryption and deep learning [J]. Journal on Communications, 2021, 42(9): 96-105.