基于参数回归的快速全景图像拼接算法

doi:10.11959/j.issn.1000-436x.2023182

Abstract

Abstract:

In reality, the field of view of images acquired by cameras was usually limited, and the demand for panoramic images was increasing.Therefore, a fast panoramic image stitching algorithm based on parameter regression was proposed for panoramic image sequences.The traditional image registration task was transformed into deep learning combined with machine learning, a multi-scale deep convolutional neural network (MDCNN) based on Gaussian difference pyramid was designed to extract features of stitching images, and LightGBM regression model was used to predict stitching parameters.The transformation matrix and the focal length of the camera were obtained to align the images, and a hyperbolic image fusion algorithm was designed to eliminate the stitching seam between the images.The experimental results show that the proposed algorithm can quickly mosaic images and obtain clearer and more natural panoramic mosaic effects than the existing representative algorithms.It also has good adaptability for infrared images.

Key words: image stitching, panoramic image, feature extraction, parameter regression, image fusion

CLC Number:

TP391.41

Fan GUO, Xiaohu LI, Wentao LIU, Jin TANG. Fast panoramic image stitching algorithm based on parameter regression[J]. Journal on Communications, 2023, 44(9): 36-47.

Figures/Tables 18

References 17

[1]	HARRIS C , STEPHENS M . A combined corner and edge detector[C]// Proceedings of the Alvey Vision Conference 1988.[S.l.]: Alvey Vision Club, 1988: 147-151.
[2]	SHU T . Multi-sensor remote sensing image alignment based on fast algorithms[J]. Journal of Intelligent Systems, 2023,32(1): 1-13.
[3]	BROWN M , LOWE D G . Automatic panoramic image stitching using invariant features[J]. International Journal of Computer Vision, 2007,74(1): 59-73.
[4]	ZARAGOZA J , CHIN T J , BROWN M S ,et al. As-projectiveas-possible image stitching with moving DLT[C]// Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2013: 2339-2346.
[5]	CHANG C H , SATO Y , CHUANG Y Y . Shape-preserving half-projective warps for image stitching[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 3254-3261.
[6]	LIN C C , PANKANTI S U , RAMAMURTHY K N ,et al. Adaptive as-natural-as-possible image stitching[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2015: 1155-1163.
[7]	CHEN Y S , CHUANG Y Y . Natural image stitching with the global similarity prior[C]// European Conference on Computer Vision. Berlin:Springer, 2016: 186-201.
[8]	曾国奇, 牛子凡, 郑丽丽 ,等. 基于球形变换的无人机视频图像实时拼接方法[J]. 航空学报, 2023,44(23): 1-11.
	ZENG G Q , NIU Z F , ZHEN L L ,et al. A real time video image stitching method for UAV based on spherical transformation[J]. Acta Aeronautica et Astronautica Sinica, 2023,44(23): 1-11.
[9]	ZHU S D , ZHANG Y Z , ZHANG J ,et al. ISGTA:an effective approach for multi-image stitching based on gradual transformation matrix[J]. Signal,Image and Video Processing, 2023,17(7): 3811-3820.
[10]	KHAMIYEV I , ISSA D , AKHTAR Z ,et al. Panoramic image generation using deep neural networks[J]. Soft Computing, 2023,27(13): 8679-8695.
[18]	KE G , MENG Q , FINLEY T ,et al. LightGBM:a highly efficient gradient boosting decision tree[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York:ACM Press, 2017: 3149-3157.
[19]	FISCHLER M A , BOLLES R C . Random sample consensuss:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981,24(6): 381-395.
[20]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[21]	IANDOLA F N , HAN S , MOSKEWICZ M W ,et al. SqueezeNet:AlexNet-level accuracy with 50x fewer parameters and ＜0.5MB model size[J]. arXiv Preprint,arXiv:1602.07360, 2016.
[22]	IANDOLA F , MOSKEWICZ M , KARAYEV S ,et al. Densenet:Implementing efficient convnet descriptor pyramids[J]. arXiv Preprint,arXiv:1404.1869, 2014.
[23]	HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016: 770-778.
[24]	MITTAL A , SOUNDARARAJAN R , BOVIK A C . Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013,20(3): 209-212.
[25]	BLAU Y , MECHREZ R , TIMOFTE R ,et al. The 2018 PIRM challenge on perceptual image super-resolution[C]// European Conference on Computer Vision. Berlin:Springer, 2019: 334-355.
[11]	DAI Q Y , FANG F M , LI J C ,et al. Edge-guided composition network for image stitching[J]. Pattern Recognition, 2021,118:108019.
[12]	ZHANG H , ZHANG J M , PERAZZI F ,et al. Deep image compositing[C]// Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway:IEEE Press, 2021: 365-374.
[13]	LU C N , CHANG Y C , CHIU W C . Bridging the visual gap:wide-range image blending[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2021: 843-851.
[14]	PARIS S , KORNPROBST P , TUMBLIN J ,et al. Bilateral filtering:theory and applications[J]. Foundations and Trends in Computer Graphics and Vision, 2008,4(1): 1-75.
[15]	SHLENS J . A tutorial on principal component analysis[J]. arXiv Preprint,arXiv:1404.1100, 2014.
[16]	FRIEDMAN J H . Greedy function approximation:a gradient boosting machine[J]. The Annals of Statistics, 2001,29(5): 1189-1232.
[17]	CHEN T Q , GUESTRIN C . XGBoost:a scalable tree boosting system[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2016: 785-794.

Metrics

Recommended 0

No Suggested Reading articles found!

分类网络	Top1准确率	Top5准确率
AlexNet	56.522%	79.066%
SqueezeNet	58.092%	80.420%
DenseNet	74.434%	91.972%
VGG16	73.360%	91.516%
VGG19	74.218%	91.842%
ResNet	73.314%	91.420%
MDCNN（不引入DoGPM）	74.998%	91.807%
MDCNN（不引入衍生图）	75.318%	92.684%
MDCNN（引入DoGPM和衍生图）	75.683%	92.728%

算法	MAE	RMSE	R²
Linear	0.098	0.127	0.784
RandomForest	0.028	0.038	0.822
SVM	0.036	0.040	0.835
KNN	0.026	0.037	0.579
XGBoost	0.029	0.043	0.804
BP	0.173	0.200	0.753
LightGBM	0.022	0.035	0.864

算法	NIQE	PI	拼接成功率	运行时间/s
AutoStitch^[3]	4.411	4.358	90%	5.146
APAP^[4]	6.163	7.195	60%	71.632
AANAP^[6]	5.322	5.976	80%	46.249
SPHP^[5]	5.675	6.732	85%	62.751
NISWGSP^[7]	4.831	5.368	95%	12.843
本文算法	4.372	4.373	100%	4.386

算法	拼接成功率	运行时间/s
AutoStitch	90%	36.375
APAP	25%	183.276
AANAP	60%	123.164
SPHP	45%	146.527
NISWGSP	80%	80.892
本文算法	100%	8.184

Fast panoramic image stitching algorithm based on parameter regression

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 17

Related Articles 15

Metrics

Recommended 0

[1]	Feibo JIANG, Yubo PENG, Li DONG. Deep image semantic communication model for 6G [J]. Journal on Communications, 2023, 44(3): 198-208.
[2]	Julong LAN, Di ZHU, Dan LI. Intelligent prediction method of virtual network function resource capacity for polymorphic network service slicing [J]. Journal on Communications, 2022, 43(6): 143-155.
[3]	Xiaodan WANG, Jingtai LI, Yafei SONG. DDAC: a feature extraction method for model of image steganalysis based on convolutional neural network [J]. Journal on Communications, 2022, 43(5): 68-81.
[4]	Jie LAI, Xiaodan WANG, Qian XIANG, Yafei SONG, Wen QUAN. Review on autoencoder and its application [J]. Journal on Communications, 2021, 42(9): 218-230.
[5]	Limin XIAO,Xiangrong XU,Zhuangkun WEI,Shenghan LIU,Yiwen LIU. Channel impulse response insensitive feature for non-coherent signal detection in molecular communication [J]. Journal on Communications, 2020, 41(9): 49-58.
[6]	Chunxiang GU,Weisen WU,Ya’nan SHI,Guangsong LI. Method of unknown protocol classification based on autoencoder [J]. Journal on Communications, 2020, 41(6): 88-97.
[7]	Jingyi QU,Meng YE,Xing QU. Airport delay prediction model based on regional residual and LSTM network [J]. Journal on Communications, 2019, 40(4): 149-159.
[8]	Shan GAI. Feature extraction algorithm based on quaternion common spatial pattern for banknote recognition [J]. Journal on Communications, 2018, 39(12): 40-46.
[9]	Wei-guo SHEN,Wei WANG. Keystroke features recognition based on stable linear discriminant analysis [J]. Journal on Communications, 2017, 38(Z2): 26-29.
[10]	Xiao-lin XU,Xiao-chun YUN,Yong-lin ZHOU,Xue-bin KANG. Online analytical model of massive malware based on feature clusting [J]. Journal on Communications, 2013, 34(8): 146-153.
[11]	. Online analytical model of massive malware based on feature clusting [J]. Journal on Communications, 2013, 34(8): 19-153.
[12]	Jing LIAN,Ke WANG,Guang-xin LI. Edge-based image fusion algorithm with wavelet transform [J]. Journal on Communications, 2007, 28(4): 18-23.
[13]	Run-qiang YAN,Yi-sheng ZHU. Speech endpoint detection based on recurrence rate analysis [J]. Journal on Communications, 2007, 28(1): 35-39.
[14]	Li LU,Yong-tao WANG,Jin-wen TIAN,Jian LIU. Cloud removal from satellite imagery based on SUSAN algorithm [J]. Journal on Communications, 2006, 27(8): 160-164.
[15]	Xin-ran LIU. New network attack classification architecture [J]. Journal on Communications, 2006, 27(2): 160-167.