基于视觉Transformer的多损失融合水下图像增强网络

doi:10.11959/j.issn.2096-6652.202252

Abstract

Abstract:

Due to the absorption and scattering of light in water, the images captured by underwater robots suffer from color distortion and reduced contrast.Aiming at alleviating the quality degradation phenomenon of underwater images, an underwater image enhancement network based on vision Transformer that be trained with multiple losses fusion strategy was proposed.The image enhancement network adopted an encoder-decoder architecture, and could be trained in an end-to-end manner.In order to effectively update the parameters of the network for enhancing underwater images, a linear combination of various losses was adopted as the overall optimization objective, including pixel loss, structure loss, edge loss and feature loss.Quantitative experiments were carried out on two large underwater datasets, and the proposed underwater image enhancement network was compared with 7 underwater image enhancement algorithms.The full reference evaluation metrics peak signal-to-noise ratio and structural similarity were calculated in experiment, and the non-referenced metric underwater image quality measure was also computed.The experimental results showed that the proposed underwater image enhancement network could effectively deal with color distortion and contrast reduction.

Key words: underwater image, quality enhancement, vision Transformer, neural network

CLC Number:

TP391

Xiaofeng CONG,Jie GUI,Jun ZHANG. Underwater image enhancement network based on visual Transformer with multiple loss functions fusion[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 522-532.

Figures/Tables 10

指标	UDCP	IBLA	ULAP	GB	UResNet	UWNet	WaterNet	UWT-Net
PSNR	12.368	17.299	17.601	15.078	18.684	18.913	$22 . 694$	21.163
SSIM	0.526	0.678	0.692	0.608	0.754	0.784	0.854	$0 . 870$
UIQM	1.842	1.783	1.878	2.272	2.772	2.746	2.883	$3 . 084$

指标	UDCP	IBLA	ULAP	GB	UResNet	UWNet	WaterNet	UWT-Net
PSNR	14.994	22.138	21.897	16.553	27.638	27.727	23.722	$28 . 343$
SSIM	0.570	0.721	0.761	0.606	0.839	0.832	0.818	$0 . 871$
UIQM	2.114	2.214	2.369	2.583	2.867	2.888	$2 . 889$	2.806

指标	设置1	设置2	设置3	设置4
PSNR	$21 . 163$	19.244	20.577	19.973
SSIM	$0 . 870$	0.830	0.853	0.851
UIQM	$3 . 084$	2.900	3.041	3.053

References 33

[1]	严浙平, 曲思瑜, 邢文 . 水下图像增强方法研究综述[J]. 智能系统学报, 2022,17(5): 860-873.
	YAN Z P , QU S Y , XING W . An overview of underwater image enhancement methods[J]. CAAI Transactions on Intelligent Systems, 2022,17(5): 860-873.
[2]	DREWS JR P , DO NASCIMENTO E , MORAES F ,et al. Transmission estimation in underwater single images[C]// Proceedings of 2013 IEEE International Conference on Computer Vision Workshops. Piscataway:IEEE Press, 2013: 825-830.
[3]	DREWS JR P , NASCIMENTO E R , BOTELHO S S C ,et al. Underwater depth estimation and image restoration based on single images[J]. IEEE Computer Graphics and Applications, 2016,36(2): 24-35.
[4]	PENG Y T , COSMAN P C . Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017,26(4): 1579-1594.
[5]	SONG W , WANG Y , HUANG D M ,et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]// Proceedings of 2018 Advances in Multimedia Information Processing. Cham:Springer, 2018: 678-688.
[6]	LIU P , WANG G Y , QI H ,et al. Underwater image enhancement with a deep residual framework[J]. IEEE Access, 2019,7: 94614-94629.
[7]	NAIK A , SWARNAKAR A , MITTAL K . Shallow-UWnet:compressed model for underwater image enhancement (student abstract)[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(18): 15853-15854.
[8]	LI C Y . Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020,98.
[9]	BERMAN D , LEVY D , AVIDAN S ,et al. Underwater single image color restoration using haze-lines and a new quantitative dataset[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(8): 2822-2837.
[10]	LIU Z , LIN Y T , CAO Y ,et al. Swin Transformer:hierarchical vision transformer using shifted windows[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 9992-10002.
[11]	ZAMIR S W , ARORA A , KHAN S ,et al. Restormer:efficient transformer for high-resolution image restoration[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2022: 5718-5729.
[12]	DOSOVITSKIY A , BEYER L , KOLESNIKOV A ,et al. An image is worth 16×16 words:transformers for image recognition at scale[J]. arXiv preprint, 2020,arXiv:2010.11929.
[13]	LI J , SKINNER K A , EUSTICE R M ,et al. WaterGAN:unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018,3(1): 387-394.
[14]	常戬, 韩旭 . 结合导向滤波与自适应算子的水下图像增强[J]. 计算机工程与应用, 2021,已录用.
	CHANG J , HAN X . Underwater image enhancement combining guide filtering with adaptive operator[J]. Computer Engineering and Applications, 2021,Accepted.
[15]	李颖, 陈龙, 黄钊宏 ,等. 基于多尺度卷积神经网络特征融合的植株叶片检测技术[J]. 智能科学与技术学报, 2021,3(3): 304-311.
	LI Y , CHEN L , HUANG Z H ,et al. Plant leaf detection technology based on multi-scale CNN feature fusion[J]. Chinese Journal of Intelligent Science and Technology, 2021,3(3): 304-311.
[16]	王禾扬, 杨启鸣, 朱旗 . 基于深度卷积集成网络的视网膜多种疾病筛查和识别方法[J]. 智能科学与技术学报, 2021,3(3): 259-267.
	WANG H Y , YANG Q M , ZHU Q . Retinal multi-disease screening and recognition method based on deep convolution ensemble network[J]. Chinese Journal of Intelligent Science and Technology, 2021,3(3): 259-267.
[17]	邵虹, 张鸣坤, 崔文成 . 基于分层卷积神经网络的皮肤镜图像分类方法[J]. 智能科学与技术学报, 2021,3(4): 474-481.
	SHAO H , ZHANG M K , CUI W C . Classification method of dermoscopic image based on hierarchical convolution neural network[J]. Chinese Journal of Intelligent Science and Technology, 2021,3(4): 474-481.
[18]	陈龙, 丁丹丹 . 多残差联合学习的水下图像增强[J]. 中国图象图形学报, 2022,27(5): 1577-1588.
	CHEN L , DING D D . Joint multi-residual learning for underwater image enhancement[J]. Journal of Image and Graphics, 2022,27(5): 1577-1588.
[19]	范新南, 杨鑫, 史鹏飞 ,等. 特征融合生成对抗网络的水下图像增强[J]. 计算机辅助设计与图形学学报, 2022,34(2): 264-272.
	FAN X N , YANG X , SHI P F ,et al. Underwater image enhancement based on feature fusion generative adversaral networks[J]. Journal of Computer-Aided Design ＆ Computer Graphics, 2022,34(2): 264-272.
[20]	LI C Y , GUO C L , REN W Q ,et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020,29: 4376-4389.
[21]	GUO Y C , LI H Y , ZHUANG P X . Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020,45(3): 862-870.
[22]	FABBRI C , ISLAM M J , SATTAR J . Enhancing underwater imagery using generative adversarial networks[C]// Proceedings of 2018 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2018: 7159-7165.
[23]	BOUDIAF A , GUO Y H , GHIMIRE A ,et al. Underwater image enhancement using pre-trained transformer[C]// Proceedings of International Conference on Image Analysis and Processing.[S.l.:s.n.], 2022: 480-488.
[24]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[J]. arXiv preprint, 2017,arXiv:1706.03762.
[25]	SONG Y D , HE Z Q , QIAN H ,et al. Vision transformers for single image dehazing[J]. arXiv preprint, 2022,arXiv:2204.03883.
[26]	ZHAO H , GALLO O , FROSIO I ,et al. Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017,3(1): 47-57.
[27]	ZHANG H , PATEL V M . Densely connected pyramid dehazing network[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 3194-3203.
[28]	JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution[C]// Proceedings of 2016 European Conference on Computer Vision.[S.l.:s.n.], 2016: 694-711.
[29]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint,2014 2014,arXiv:1409.
[30]	ISLAM M J , XIA Y Y , SATTAR J . Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020,5(2): 3227-3234.
[31]	LI C Y , QUO J , PANG Y W ,et al. Single underwater image restoration by blue-green channels dehazing and red channel correction[C]// Proceedings of 2016 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2016: 1731-1735.
[32]	WANG Y , SONG W , FORTINO G ,et al. An experimental-based review of image enhancement and image restoration methods for underwater imaging[J]. IEEE Access, 7: 140233-140251.
[33]	PANETTA K , GAO C , AGAIAN S . Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016,41(3): 541-551.

Metrics

Recommended 0

No Suggested Reading articles found!

Underwater image enhancement network based on visual Transformer with multiple loss functions fusion

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 33

Related Articles 13

Metrics

Recommended 0

[1]	Xinhuan LUO, Yixuan WANG, Wei LI, Xi CHEN. Application of Fit CutMix data augmentation algorithm based on saliency information in medical images [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 58-68.
[2]	Zhouyu GU, Yuecheng YU, Tiantian Zhe. Rapider-YOLOX: lightweight object detection network with high precision [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 92-103.
[3]	Jingwei LU, Xiang CHENG, Fei-Yue WANG. Artificial intelligence and deep learning methods for solving differential equations: the state of the art and prospects [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 461-476.
[4]	Yongqiang ZHANG, Meilin SONG, Tianhu LIU, Menghua MAN. Research on three frame difference gesture recognition method based on mixed bone features [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 592-599.
[5]	Yuanqi QIN, Qingyuan JI, Jun GE, Xingyuan DAI, Yuanyuan CHEN, Xiao WANG. Short-term traffic state reasoning and precise prediction in urban networks [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 380-395.
[6]	Hong SHAO, Mingkun ZHANG, Wencheng CUI. Classification method of dermoscopic image based on hierarchical convolution neural network [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 474-481.
[7]	Yang ZHANG, Yue HU, Dongrong XIN. A deep learning short-term traffic flow prediction method considering spatial-temporal association [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 172-178.
[8]	Hanbing GAO, Chi ZHANG, Mingyan JIN, Yang XIAO, Fengyu CONG. Cross-subject emotional EEG recognition based on multi-source domain adaptation [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(1): 59-64.
[9]	Limin QUAN,Cuili YANG,Junfei QIAO. CFNN-based online control for dissolved oxygen concentration of wastewater treatment processes [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(3): 261-267.
[10]	Guobin ZHANG,Xinying WANG. Research on data-driven modeling for photovoltaic characteristics based on hybrid neural network [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(2): 169-178.
[11]	Xiaochen HAO,Yakun JI,Lizhao ZHENG,Xin SHI,Yantao ZHAO. Dynamic optimization algorithm of cement firing system based on differential evolution [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(2): 186-193.
[12]	Dongyuan LIN,Xiaofeng CHEN,Wentao SUN,Weikai LI,Yannan XIA. Stability analysis of commutative quaternion valued neural network with time varying delays [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(1): 80-87.
[13]	Peifeng NIU,Yunpeng MA,Xinxin ZHANG,Xiaobin HU. Research and application on combustion optimization of coal-fired boiler in thermal power plant based on artificial intelligence technology [J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(2): 163-170.