基于强化样本的伪孪生网络图像篡改定位模型

doi:10.11959/j.issn.2096-109x.2024010

Abstract

Abstract:

With the continuous development of the internet, an increasing number of images have been tampered with on the network, accompanied by a growing range of techniques to cover up tampering traces.However, most current detection models neglect the impact of image post-processing on tamper detection algorithms, limiting their real-life applications.To address these issues, a general image tampering location model based on enhanced samples and the pseudo-twin network was proposed.The pseudo-twin network enabled the model to learn tampering features in real images.On one hand, by applying convolution constraints, the image content was suppressed, allowing the model to focus more on residual trace information of tampering.The two-branch structure of the network facilitated the comprehensive utilization of image feature information.By utilizing enhanced samples, the model could dynamically generate the most crucial pictures for learning tamper types, enabling targeted training of the model.This approach ensured that the model converged in all directions, ultimately obtaining the global optimal model.The idea of data enhancement was employed to automatically generate abundant tampered images and corresponding masks, effectively resolving the limited tampering dataset issue.Extensive experiments were conducted on four datasets, demonstrating the feasibility and effectiveness of the proposed model in pixel-level tamper detection.Particularly on the Columbia dataset, the algorithm achieves a 33.5% increase in F1 score and a 23.3% increase in MCC score.These results indicate that the proposed model harnesses the advantages of deep learning models and significantly improves the effectiveness of tamper location detection.

Key words: enhanced sample, tampering positioning, pseudo-siamese network, data augmentation, tampering image

CLC Number:

TP391

Jinwei WANG, Zihe ZHANG, Xiangyang LUO, Bin MA. Pseudo-siamese network image tampering localization model based on reinforced samples[J]. Chinese Journal of Network and Information Security, 2024, 10(1): 33-47.

Figures/Tables 20

References 30

[1]	FERRARA P , BIANCHI T , DE ROSA A ,et al. Image forgery localization via fine-grained analysis of CFA artifacts[J]. IEEE Transactions on Information Forensics and Security, 2012,7(5): 1566-1577.
[2]	LI C , MA Q , XIAO L M ,et al. An image copy move forgery detection method using QDCT[C]// Proceedings of the International Conference on Internet Multimedia Computing and Service. 2016: 5-8.
[3]	AMBILI B , GEORGE N . A robust technique for splicing detection in tampered blurred images[C]// Proceedings of 2017 International Conference on Trends in Electronics and Informatics (ICEI). 2017: 897-901.
[4]	黄维, 黄添强, 张雪莉 ,等. 基于块效应网格偏移的重获取JPEG图像篡改检测[J]. 网络与信息安全学报, 2017,3(12): 24-30.
	HUANG W , HUANG T Q , ZHANG X L ,et al. JPEG recapture image tamper detection method based on block effect grid offset[J]. Chinese Journal of Network and Information Security, 2017,3(12): 24-30.
[5]	BAYAR B , STAMM M C . Constrained convolutional neural networks:a new approach towards general purpose image manipulation detection[J]. IEEE Transactions on Information Forensics and Security, 2018,13(11): 2691-2706.
[6]	WU Y , ABD-ALMAGEED W , NATARAJAN P . Image copy-move forgery detection via an end-to-end deep neural network[C]// Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). 2018: 1907-1915.
[7]	蒋小玉, 刘春晓 . 边缘与区域不一致性引导下的图像拼接篡改检测网络[J]. 中国图象图形学报, 2021,26(10): 2411-2420.
	JIANG X Y , LIU C X . Edge and region inconsistency-guided image splicing tamper detection network[J]. Journal of Image and Graphics, 2021,26(10): 2411-2420.
[8]	朱新同, 唐云祁, 耿鹏志 . 基于特征融合的篡改与深度伪造图像检测算法[J]. 信息网络安全, 2021(8): 70-81.
	ZHU X T , TANG Y Q , GENG P Z . Detection algorithm of tamper and deepfake image based on feature fusion[J]. Netinfo Security, 2021(8): 70-81.
[9]	李小红, 王晓霞 . 一种剪枝压缩CNN移动设备端图像篡改检测方法[J]. 计算机仿真, 2021,38(3): 83-86,91.
	LI X H , WANG X X . An image tamper detection method for CNN mobile devices based on pruning compression[J]. Computer Simulation, 2021,38(3): 83-86,91.
[10]	何家豪 . 图像复制移动篡改检测研究及实现[D]. 大连:大连理工大学, 2021.
	HE J H . Research and implementation of image copy mobile tampering detection[D]. Dalian:Dalian University of Technology, 2021.
[11]	ZHUANG P Y , LI H D , TAN S Q ,et al. Image tampering localization using a dense fully convolutional network[J]. IEEE Transactions on Information Forensics and Security, 2021,16: 2986-2999.
[12]	WEI Y , MA J F , WANG Z Z ,et al. Image splicing forgery detection by combining synthetic adversarial networks and hybrid dense U-net based on multiple spaces[J]. International Journal of Intelligent Systems, 2022,37(11): 8291-8308.
[13]	LU W , XU W B , SHENG Z Q . An interpretable image tampering detection approach based on cooperative game[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023,33(2): 952-962.
[14]	ZHAO T C , XU X , XU M Z ,et al. Learning self-consistency for deepfake detection[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021: 15003-15013.
[15]	CHEN L , ZHANG Y , SONG Y B ,et al. Self-supervised learning of adversarial example:towards good generalizations for deepfake detection[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022: 18689-18698.
[16]	SOLIMAN M , LEHMAN C , ALREGIB G . S6:semi-supervised self-supervised semantic segmentation[C]// Proceedings of 2020 IEEE International Conference on Image Processing (ICIP). 2020: 1861-1865.
[17]	XU H , LI X , GONG W Y . Research on recognition of landslides with remote sensing images based on extreme learning machine[C]// Proceedings of 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC). 2017: 740-747.
[18]	HALIASSOS A , MIRA R , PETRIDIS S ,et al. Leveraging real talking faces via self-supervision for robust forgery detection[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022: 14930-14942.
[19]	KRISHNA V A , REDDY A , NAGAJYOTHI D . Signature recognition using Siamese neural networks[C]// Proceedings of 2021 IEEE International Conference on Mobile Networks and Wireless Communications (ICMNWC). 2022: 1-4.
[20]	WANG Y Y , ZHANG W S , ZHANG L M ,et al. Depthwise over-parameterized Siamese network for visual tracking[C]// Proceedings of 2021 International Conference on Information Technology and Biomedical Engineering (ICITBE). 2022: 58-62.
[21]	SUN K , YAO T P , CHEN S ,et al. Dual contrastive learning for general face forgery detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022,36(2): 2316-2324.
[22]	COZZOLINO D , POGGI G , VERDOLIVA L . Splicebuster:a new blind image splicing detector[C]// Proceedings of 2015 IEEE International Workshop on Information Forensics and Security (WIFS). 2015: 1-6.
[23]	ZHOU P , HAN X T , MORARIU V I ,et al. Learning rich features for image manipulation detection[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018: 1053-1061.
[24]	ZHAO Q , CAO G , ZHOU A T ,et al. Image tampering detection via semantic segmentation network[C]// Proceedings of 2020 15th IEEE International Conference on Signal Processing (ICSP). 2020: 165-169.
[25]	CHEN J X , LIAO X , WANG W ,et al. SNIS:a signal noise separation-based network for post-processed image forgery detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023,33(2): 935-951.
[26]	WU Y , ABDALMAGEED W , NATARAJAN P . ManTra-Net:manipulation tracing network for detection and localization of image forgeries with anomalous features[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 9535-9544.
[27]	CHEN X R , DONG C B , JI J Q ,et al. Image manipulation detection by multi-view multi-scale supervision[C]// Proceedings of International Conference on Computer Vision. 2021.
[28]	WANG J , WU Z , CHEN J ,et al. ObjectFormer for image manipulation detection and localization[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2022.
[29]	DONG J , WANG W , TAN T N . CASIA image tampering detection evaluation database[C]// Proceedings of 2013 IEEE China Summit and International Conference on Signal and Information Processing. 2013: 422-426.
[30]	HSU Y F , CHANG S F . Detecting image splicing using geometry invariants and camera characteristics consistency[C]// Proceedings of IEEE International Conference on Multimedia and Expo. 2006.

Metrics

Recommended 0

No Suggested Reading articles found!

后处理操作类型	CASIA测试集F1值
JPEG压缩（质量因子Q=80）	0.532
高斯模糊（半径r=2）	0.534
边缘增强（Filteraegs=(3,3)，2,0）	0.537
改变色度（增强因子=0.7）	0.541
对比度增强（增强因子=1.5）	0.535

后处理操作	CASIA测试集F1值
高斯模糊（r=2）
均值滤波（r=6）
边缘增强（Filteraegs=(3,3),2,0）
锐化（Filteraegs=(3,3),16,0）
平滑（Filteraegs=(5,5),100,0）	0.542
细节提升（Filteraegs=(3,3),1,255）
改变亮度（增强因子=0.8）
改变色度（增强因子=0.7）
对比度增强（增强因子=1.5）
锐度增强（增强因子=3.0）
高斯模糊（r=4）
均值滤波（r=10）
边缘增强（Filteraegs=(3,3),2,0）
锐化（Filteraegs=(3,3),16,0）
平滑（Filteraegs=(5,5),100,0）	0.532
细节提升（Filteraegs=(3,3),1,255）
改变亮度（增强因子=0.9）
改变色度（增强因子=0.8）
对比度增强（增强因子=1.6）
锐度增强（增强因子=3.1）
高斯模糊（r=6）
均值滤波（r=12）
边缘增强（Filteraegs=(3,3),2,0）
锐化（Filteraegs=(3,3),16,0）
平滑（Filteraegs=(5,5),100,0）	0.538
细节提升（Filteraegs=(3,3),1,255）
改变亮度（增强因子=1.0）
改变色度（增强因子=0.9）
对比度增强（增强因子=1.7）
锐度增强（增强因子=3.2）

训练集中CASIA:自制数据集	训练集中CASIA图像数量	训练集中制作图像数量	测试集CASIAF1值
0.90:0.10	4 965	551	0.541
0.85:0.15	4 689	827	0.535
0.80:0.20	4 413	1 103	0.539
0.75:0.25	4 137	1 379	0.537
0.70:0.30	3 861	1 655	0.532
0.60:0.40	3 309	2 207	0.526
0.40:0.60	2 206	3 310	0.531
0.30:0.70	1 654	3 862	0.527

数据集	原图流	约束卷积流	强化样本模块	F1值
	√			0.458
CASIA	√	√		0.492
	√	√	√	0.542
	√			0.598
DSO-1	√	√		0.634
	√	√	√	0.683
	√			0.846
Columbia	√	√		0.894
	√	√	√	0.973

指导流	CASIA F1值
强化样本+原图流预测分类	0.542
强化样本+约束卷积流预测分类	0.528
强化样本+融合流预测分类	0.534

Pseudo-siamese network image tampering localization model based on reinforced samples

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 20

References 30

Related Articles 2

Metrics

Recommended 0

训练集中NIST16:自制数据集	训练集中NIST16图像数量	训练集中制作图像数量	测试集NIST16F1值
0.10:0.90	150	1353	0.628
0.20:0.80	300	1203	0.631
0.30:0.70	451	1052	0.625

数据集	F1值	MCC	召回率	精确率
CASIA	0.542	0.512	0.506	0.584
DSO-1	0.683	0.573	0.678	0.689
NIST16	0.635	0.572	0.639	0.631
Columbia	0.973	0.947	0.957	0.988

模型	CASIA		Columbia
模型	F1值	MCC	F1值	MCC
NOI3^[22]	0.176	0.089	0.448	0.207
CFA2^[1]	0.212	0.162	0.503	0.328
SNIS^[25]	0.368	—	0.681	—
RGB-N^23]	0.408	0.287	0.697	—
Zhao^[24]	0.423	0.427	0.733	0.714
ManTra^[26]	0.46	0.45	0.793	0.742
MVSS-Net^[27]	0.452	0.404	0.638	—
Lu^[13]	0.51	0.49	—	—
DenFCN^[11]	0.559	0.516	—	—
Wang^[28]	0.579	0.518	—	—
本文模型	0.542	0.512	0.973	0.947

[1]	Guanyun FENG, Cai FU, Jianqiang LYU, Lansheng HAN. Insider threat detection based on operational attention and data augmentation [J]. Chinese Journal of Network and Information Security, 2023, 9(3): 102-112.
[2]	Yuchao JIANG,Lixin JI,Chao GAO,Shaomei LI. Research on synthesis data generation method for logo recognition [J]. Chinese Journal of Network and Information Security, 2018, 4(5): 21-31.