载体独立的抗屏摄信息膜叠加水印算法

doi:10.11959/j.issn.2096-109x.2023045

网络与信息安全学报 ›› 2023, Vol. 9 ›› Issue (3): 135-149.doi: 10.11959/j.issn.2096-109x.2023045

载体独立的抗屏摄信息膜叠加水印算法

李晓萌¹, 郭玳豆¹, 卓训方², 姚恒¹, 秦川¹

¹ 上海理工大学光电信息与计算机工程学院，上海 200093
² 上海数据交易所有限公司，上海 201203

修回日期:2023-04-23 出版日期:2023-06-25 发布日期:2023-06-01
作者简介:李晓萌（1997- ），女，安徽淮北人，上海理工大学硕士生，主要研究方向为深度学习和信息隐藏
郭玳豆（1993- ），男，河南安阳人，上海理工大学博士生，主要研究方向为深度学习和数字水印
卓训方（1973- ），男，上海人，博士，上海数据交易所有限公司研究员，主要研究方向为数据流通与安全
姚恒（1982- ），男，安徽芜湖人，博士，上海理工大学副教授，主要研究方向为多媒体信息安全和模式识别
秦川（1980- ），男，安徽芜湖人，博士，上海理工大学教授、博士生导师，主要研究方向为多媒体信息安全和AI安全
基金资助:
国家自然科学基金(U20B2051);国家自然科学基金(62172280);国家自然科学基金(62172281);上海市自然科学基金(21ZR1444600)

Carrier-independent screen-shooting resistant watermarking based on information overlay superimposition

Xiaomeng LI¹, Daidou GUO¹, Xunfang ZHUO², Heng YAO¹, Chuan QIN¹

¹ School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
² Shanghai Data Exchange Corporation, Shanghai 201203, China

Revised:2023-04-23 Online:2023-06-25 Published:2023-06-01
Supported by:
The National Natural Science Foundation of China(U20B2051);The National Natural Science Foundation of China(62172280);The National Natural Science Foundation of China(62172281);The National Natural Science Foundation of Shanghai(21ZR1444600)

摘要/Abstract

摘要：

金融安全作为国家安全的重要组成部分，是经济平稳健康发展的重要基础。数字图像水印技术在金融信息安全方面发挥着巨大作用，其中，抗屏幕拍摄水印算法成为数字图像水印技术新的研究重点。如何兼顾水印图像的视觉质量和鲁棒性是抗屏幕拍摄鲁棒水印算法需要解决的重要问题。现有的水印方案一般通过对载体图像进行修改来实现水印不可见的目的，这种水印嵌入方式不具有普适性。为此，提出了一种新的基于深度学习的端到端抗屏幕拍摄鲁棒水印算法。作为端到端网络架构的一部分，编码器网络的输入是随机二进制字符串，经过网络训练后根据输入的水印信息生成相应的水印信息膜，可以附加在任意的载体图像上。使用数学方法模拟了屏幕拍摄过程中可能产生的失真，使得经过网络学习后的模型具有抵抗屏幕拍摄噪声的能力。增加了基于图像的恰可察觉差损失来进一步提升水印图像的视觉质量。此外，为了更灵活地平衡水印图像的视觉质量和鲁棒性，在训练阶段设计了一个嵌入超参数，通过改变嵌入超参数的大小，就可以得到适合不同场景的模型。为了验证所提算法的有效性，针对视觉质量和鲁棒性做了多种实验。实验结果表明，与目前的主流算法相比，使用所提算法生成的水印图像具有更好的视觉质量，并且在不同距离、不同角度、不同光照条件和不同设备的鲁棒性实验中均可以准确还原原始水印信息。

关键词: 深度学习, 鲁棒水印, 屏幕拍摄, 视觉质量, 恰可察觉差

Abstract:

Financial security, an important part of national security, is critical for the stable and healthy development of the economy.Digital image watermarking technology plays a crucial role in the field of financial information security, and the anti-screen watermarking algorithm has become a new research focus of digital image watermarking technology.The common way to achieve an invisible watermark in existing watermarking schemes is to modify the carrier image, which is not suitable for all types of images.To solve this problem, an end-to-end robust watermarking scheme based on deep learning was proposed.The algorithm achieved both visual quality and robustness of the watermark image.A random binary string served as the input of the encoder network in the proposed end-to-end network architecture.The encoder can generate the watermark information overlay, which can be attached to any carrier image after training.The ability to resist screen shooting noise was learned by the model through mathematical methods incorporated in the network to simulate the distortion generated during screen shooting.The visual quality of the watermark image was further improved by adding the image JND loss based on just perceptible difference.Moreover, an embedding hyperparameter was introduced in the training phase to balance the visual quality and robustness of the watermarked image adaptively.A watermark model suitable for different scenarios can be obtained by changing the size of the embedding hyperparameter.The visual quality and robustness performance of the proposed scheme and the current state-of-the-art algorithms were evaluated to verify the effectiveness of the proposed scheme.The results show that the watermark image generated by the proposed scheme has better visual quality and can accurately restore the embedded watermark information in robustness experiments under different distances, angles, lighting conditions, display devices, and shooting devices.

Key words: deep learning, robust watermarking, screen-shooting, visual quality, just noticeable difference

中图分类号:

TP393

李晓萌, 郭玳豆, 卓训方, 姚恒, 秦川. 载体独立的抗屏摄信息膜叠加水印算法[J]. 网络与信息安全学报, 2023, 9(3): 135-149.

Xiaomeng LI, Daidou GUO, Xunfang ZHUO, Heng YAO, Chuan QIN. Carrier-independent screen-shooting resistant watermarking based on information overlay superimposition[J]. Chinese Journal of Network and Information Security, 2023, 9(3): 135-149.

图/表 19

图1

图2

图3

图4

图5

图6

图7

表1

图8

表2

不同数字噪声攻击下的提取准确率Table 2 Extraction accuracy with different digital noise attacks"

攻击类型	攻击强度	本文算法
攻击类型	攻击强度	$ε_{1}$	$ε_{2}$
	4/5	97.15%	96.12%
	3/4	97.15%	95.15%
缩放攻击	3/5	97.18%	97.09%
（缩放比例）	1/2	97.00%	94.17%
	2/5	96.70%	94.17%
	1/4	96.42%	93.20%
	1/5	95.55%	89.47%
	13	97.03%	96.12%
高斯模糊	15	96.54%	96.12%
（高斯核大小）	17	95.18%	92.23%
	19	93.05%	80.58%
	21	90.96%	67.96%
	40	96.30%	90.29%
	50	96.63%	94.17%
JPEG压缩	60	96.75%	96.12%
（质量因子大小）	70	96.69%	96.12%
	80	96.91%	95.15%
	90	96.88%	97.09%
	20 × 20	97.09%	97.09%
	40 × 40	96.94%	94.17%
裁剪攻击	60 × 60	97.21%	98.06%
（裁剪区域尺寸）	80 × 80	96.91%	97.09%
	100 × 100	96.18%	97.09%
	120 × 120	94.45%	93.20%
	10 pt	95.93%	90.29%
边缘遮盖	20 pt	95.27%	87.38%
（遮盖宽度）	30 pt	95.33%	90.29%
	40 pt	94.66%	89.32%
	50 pt	94.57%	82.52%

表2

图9

图10

表3

不同拍摄距离下的提取准确率Table 3 Extraction accuracy with different shooting distances"

拍摄距离/cm	$ε_{2}$
拍摄距离/cm	StegaStamp^[18]	提出的算法
20	83.33%	100.00%
35	100.00%	100.00%
55	100.00%	100.00%

表3

表4

不同拍摄角度下的提取准确率Table 4 Extraction accuracy with different shooting angles"

拍摄角度	$ε_{1}$	$ε_{1}$	$ε_{1}$	$ε_{1}$	$ε_{2}$	$ε_{1}$	$ε_{2}$
拍摄角度	Kang等^[30]	Pramila等^[4]	Nakamura等^[31]	StegaStamp^[18]		本文算法
+45°	58.89%	55.28%	83.69%	99.22%	100.00%	97.19%	100.00%
+30°	72.17%	57.00%	86.81%	99.22%	100.00%	97.19%	100.00%
+15°	80.47%	54.88%	77.73%	99.48%	100.00%	97.19%	100.00%
0°	67.50%	55.95%	84.56%	99.74%	100.00%	96.56%	100.00%
-15°	74.31%	64.03%	86.13%	99.74%	100.00%	98.13%	100.00%
-30°	73.52%	63.48%	83.69%	99.74%	100.00%	98.13%	100.00%
-45°	80.08%	71.39%	84.17%	98.69%	100.00%	97.19%	100.00%

表4

图11

图12

表5

不同光照环境下的提取准确率Table 5 Extraction accuracy with different illumination"

光照强度/lux	$ε_{1}$	$ε_{2}$	$ε_{1}$	$ε_{2}$
光照强度/lux	StegaStamp^[18]		本文算法
50	100.00%	100.00%	98.75%	100.00%
150	99.74%	100.00%	96.56%	100.00%
500	99.74%	100.00%	98.75%	100.00%

表5

表6

不同设备下的提取准确率Table 6 Extraction accuracy with different devices"

设备	名称	$ε_{1}$	$ε_{2}$
	iPhone XS	96.56%	100%
拍摄设备	Xiaomi 11	97.44%	100%
	HUAWEI P40 Pro+	98.18%	100%
	MacBook Pro 2019	96.56%	100%
显示设备	Dell G15 5511	98.18%	100%
	AOC I2490PXZ	98.32%	100%

表6

图13

参考文献 16

[17]	RONNEBERGER O , FISCHER P , BROX T . U-net:convolutional networks for biomedical image segmentation[C]// 18th International Conference on Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. 2015: 234-241.
[18]	TANCIK M , MILDENHALL B , NG R . Stegastamp:invisible hyperlinks in physical photographs[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 2117-2126.
[19]	JADERBERG M , SIMONYAN K , ZISSERMAN A . Spatial transformer networks[C]// Advances in Neural Information Processing Systems. 2015:28.
[20]	WENGROWSKI , ERIC KRISTIN DANA . Light field messaging with deep photographic steganography[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 1515-1524.
[21]	FANG HAN , DONGDONG CHEN , QIDONG HUANG ,et al. Deep template-based watermarking[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020,31(4): 1436-1451.
[22]	BORACCHI G , FOI A . Modeling the performance of image restoration from motion blur[J]. IEEE Transactions on Image Processing, 2012,21(8): 3502-3517.
[23]	KUPYN O , BUDZAN V , MYKHAILYCH M ,et al. Deblurgan:blind motion deblurring using conditional adversarial networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8183-8192.
[24]	KIM W , LEE S H , SEO Y . Image fingerprinting scheme for print-and-capture model[C]// Advances in Multimedia Information Processing-PCM 2006. 2006: 106-113.
[25]	NUUTINEN M , OITTINEN P . Simulation of printing and image capture for linking applications[J]. Graphic Arts in Finland, 2009,38(1): 1-13.
[26]	SHIN R , SONG D . Jpeg-resistant adversarial images[C]// NIPS 2017 Workshop on Machine Learning and Computer Security. 2017: 1-8.
[27]	ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein generative adversarial networks[C]// International Conference on Machine Learning (PMLR). 2017: 214-223.
[28]	LIN T Y , MAIRE M , BELONGIE S ,et al. Microsoft coco:common objects in context[C]// 13th European Conference on Computer Vision–ECCV 2014. 2014: 740-755.
[29]	SUN Y , YU Y , WANG W . Moiré photo restoration using multiresolution convolutional neural networks[J]. IEEE Transactions on Image Processing, 2018,27(8): 4160-4172.
[30]	KANG X , HUANG J , ZENG W . Efficient general print-scanning resilient data hiding based on uniform log-polar mapping[J]. IEEE Transactions on Information Forensics and Security, 2010,5(1): 1-12.
[1]	李凤华 . 信息技术与网络空间安全发展趋势[J]. 网络与信息安全学报, 2015,1(1): 8-17.
	LI F H . Development trends of the information technology and cyberspace security[J]. Chinese Journal of Network and Information Security, 2015,1(1): 8-17.
[2]	靳玉红 . 大数据环境下互联网金融信息安全防范与保障体系研究[J]. 情报科学, 2018,36(12): 134-138.
	JIN Y H . Internet financial information security and protection systems in big data environment[J]. Information Science, 2018,36(2): 134-138.
[3]	王馨雅, 华光, 江昊 ,等. 深度学习模型的版权保护研究综述[J]. 网络与信息安全学报, 2022,8(2): 1-14.
	WANG X Y , HUA G , JIANG H ,et al. Survey on intellectual property protection for deep learning model[J]. Chinese Journal of Network and Information Security, 2022,8(2): 1-14.
[4]	NAKAMURA T , KATAYAMA A , YAMAMURO M ,et al. Fast watermark detection scheme for camera-equipped cellular phone[C]// Proceedings of the 3rd International Conference on Mobile and Ubiquitous Multimedia. 2004: 101-108.
[5]	PRAMILA A , KESKINARKAUS A , SEPP?NEN T . Increasing the capturing angle in print-cam robust watermarking[J]. Journal of Systems and Software, 2018,135: 205-215.
[6]	CHOU C H , LI Y C . A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile[J]. IEEE Transactions on Circuits and Systems for Video Technology, 1995,5(6): 467-476.
[7]	FANG H , ZHANG W , ZHOU H ,et al. Screen-shooting resilient watermarking[J]. IEEE Transactions on Information Forensics and Security, 2018,14(6): 1403-1418.
[8]	FANG H , ZHANG W , MA Z ,et al. A camera shooting resilient watermarking scheme for underpainting documents[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019,30(11): 4075-4089.
[9]	CUI H , BIAN H , ZHANG W ,et al. Unseencode:Invisible on-screen barcode with image-based extraction[C]// IEEE INFOCOM 2019-IEEE Conference on Computer Communications. 2019: 1315-1323.
[10]	BALUJA S . Hiding images in plain sight:deep steganography[C]// Advances in Neural Information Processing Systems, 2017,30.
[11]	BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis ＆ Machine Intelligence, 2017,(1).
[12]	TANG W , TAN S , LI B ,et al. Automatic steganographic distortion learning using a generative adversarial network[J]. IEEE Signal Processing Letters, 2017,24(10): 1547-1551.
[13]	HU D H , WANG L , JIANG W J ,et al. A novel image steganography method via deep convolutional generative adversarial networks[J]. IEEE Access, 2018,6.
[14]	VOLKHONSKIY D , NAZAROV I , BURNAEV E . Steganographic generative adversarial networks[C]// Twelfth International Conference on Machine Vision (ICMV 2019). 11433: 991-1005.
[15]	ZHU J , KAPLAN R , JOHNSON J ,et al. Hidden:Hiding data with deep networks[C]// Proceedings of the European Conference on Computer Vision (ECCV). 2018: 657-672.
[31]	PRAMILA A , KESKINARKAUS A , SEPP?NEN T . Toward an interactive poster using digital watermarking and a mobile phone camera[J]. Signal,Image and Video Processing , 2012(6): 211-222.
[16]	LIU Y , GUO M , ZHANG J ,et al. A novel two-stage separable deep learning framework for practical blind watermarking[C]// The 27th ACM International Conference. 2019.

算法	PSNR/dB	SSIM	LPIPS
StegaStamp^[18]	27.24	0.902 6	0.0511
本文算法(α=1.5)	28.64	0.967 2	0.009 9
本文算法(α=1.6)	30.56	0.968 0	0.009 3
本文算法(α=1.7)	31.76	0.974 0	0.007 5

载体独立的抗屏摄信息膜叠加水印算法

Carrier-independent screen-shooting resistant watermarking based on information overlay superimposition

在线阅读

pdf下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 19

参考文献 16

相关文章 15

Metrics

推荐阅读 0

[1]	郭学镜, 方毅翔, 赵怡, 张天助, 曾文超, 王俊祥. 基于传统引导机制的深度鲁棒水印算法[J]. 网络与信息安全学报, 2023, 9(2): 175-183.
[2]	谢绒娜, 马铸鸿, 李宗俞, 田野. 基于卷积神经网络的加密流量分类方法[J]. 网络与信息安全学报, 2022, 8(6): 84-91.
[3]	章登勇, 文凰, 李峰, 曹鹏, 向凌云, 杨高波, 丁湘陵. 基于双分支网络的图像修复取证方法[J]. 网络与信息安全学报, 2022, 8(6): 110-122.
[4]	林佳滢, 周文柏, 张卫明, 俞能海. 空域频域相结合的唇型篡改检测方法[J]. 网络与信息安全学报, 2022, 8(6): 146-155.
[5]	陈晋音, 吴长安, 郑海斌. 基于softmax激活变换的对抗防御方法[J]. 网络与信息安全学报, 2022, 8(2): 48-63.
[6]	邱宝琳, 易平. 基于多维特征图知识蒸馏的对抗样本防御方法[J]. 网络与信息安全学报, 2022, 8(2): 88-99.
[7]	李丽娟, 李曼, 毕红军, 周华春. 基于混合深度学习的多类型低速率DDoS攻击检测方法[J]. 网络与信息安全学报, 2022, 8(1): 73-85.
[8]	秦中元, 贺兆祥, 李涛, 陈立全. 基于图像重构的MNIST对抗样本防御算法[J]. 网络与信息安全学报, 2022, 8(1): 86-94.
[9]	邹德清, 李响, 黄敏桓, 宋翔, 李浩, 李伟明. 基于图结构源代码切片的智能化漏洞检测系统[J]. 网络与信息安全学报, 2021, 7(5): 113-122.
[10]	王正龙, 张保稳. 生成对抗网络研究综述[J]. 网络与信息安全学报, 2021, 7(4): 68-85.
[11]	李炳龙, 佟金龙, 张宇, 孙怡峰, 王清贤, 常朝稳. 基于TensorFlow的恶意代码片段自动取证检测算法[J]. 网络与信息安全学报, 2021, 7(4): 154-163.
[12]	谭清尹, 曾颖明, 韩叶, 刘一静, 刘哲理. 神经网络后门攻击研究[J]. 网络与信息安全学报, 2021, 7(3): 46-58.
[13]	杨路辉,白惠文,刘光杰,戴跃伟. 基于可分离卷积的轻量级恶意域名检测模型[J]. 网络与信息安全学报, 2020, 6(6): 112-120.
[14]	刘西蒙,谢乐辉,王耀鹏,李旭如. 深度学习中的对抗攻击与防御[J]. 网络与信息安全学报, 2020, 6(5): 36-53.
[15]	杜思佳,于海宁,张宏莉. 基于深度学习的文本分类研究进展[J]. 网络与信息安全学报, 2020, 6(4): 1-13.