基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法

doi:10.11959/j.issn.2096-6652.202129

Abstract

Abstract:

A method based on visual saliency for ego-motion estimation and scale estimation of the vehicle in front was proposed.Firstly, for the ego-motion estimation of vehicle-mounted camera, the visual saliency calculation method was used to detect and remove moving objects in the monocular image sequence containing noise.While considering the image texture and smooth region, the weighted saliency map was used to retain useful feature points, to improve the robustness of ego-motion estimation.Secondly, the distance of the vehicle in front was converted into a vehicle scale estimation, by integrating descriptor match and the strength of regularization match of the lie algebra to minimize loss function.The visual attention mechanism was used to get texture image block without shade, and the pixel in the image block weight to mitigate the effects of destroyed by noise pixel, so as to realize the robust and accurate scale estimation.Finally, several challenging datasets were used to analyze and verify the proposed method.The results show that the monocular camera ego-motion estimation method reaches the level of the stereo camera-based method, and the vehicle scale estimation method ensures the prediction accuracy while giving full play to the advantages of strong robustness.

Key words: visual saliency, monocular camera, motion estimation, scale estimation

CLC Number:

TP181

Mingxin AI, Tie LIU, Jing WANG, et al. A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 280-293.

Figures/Tables 16

References 35

[1]	JAEGLE A , PHILLIPS S , DANIILIDIS K . Fast,robust,continuous monocular egomotion computation[C]// Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA). Piscataway:IEEE Press, 2016: 773-780.
[2]	LOURAKIS M I A , ARGYROS A A . SBA:a software package for generic sparse bundle adjustment[J]. ACM Transactions on Mathematical Software, 2009,36(1): 1-30.
[3]	袁野, 和晓歌, 朱定坤 ,等. 视觉图像显著性检测综述[J]. 计算机科学, 2020,47(7): 84-91.
	YUAN Y , HE X G , ZHU D K ,et al. Survey of visual image saliency detection[J]. Computer Science, 2020,47(7): 84-91.
[4]	GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 3354-3361.
[5]	刘凯, 李浥东, 林伟鹏 . 车辆再识别技术综述[J]. 智能科学与技术学报, 2020,2(1): 10-25.
	LIU K , LI Y D , LIN W P . A survey on vehicle re-identification[J]. Chinese Journal of Intelligent Science and Technology, 2020,2(1): 10-25.
[6]	熊刚, 董西松, 朱凤华 ,等. 城市交通大数据技术及智能应用系统[J]. 大数据, 2015,1(4): 81-96.
	XIONG G , DONG X S , ZHU F H ,et al. Big data technologies and intelligent application system for urban transportation[J]. Big Data Research, 2015,1(4): 81-96.
[7]	KIM G , CHO J S . Vision-based vehicle detection and inter-vehicle distance estimation[C]// Proceedings of 2012 12th International Conference on Control,Automation and Systems. Piscataway:IEEE Press, 2012: 625-629.
[8]	MUKHTAR A , XIA L K , TANG T B . Vehicle detection techniques for collision avoidance systems:a review[J]. IEEE Transactions on Intelligent Transportation Systems, 2015,16(5): 2318-2338.
[9]	DAGAN E , MANO O , STEIN G P ,et al. Forward collision warning with a single camera[C]// Proceedings of 2004 IEEE Intelligent Vehicles Symposium. Piscataway:IEEE Press, 2004: 37-42.
[10]	LIU T , YUAN Z J , SUN J ,et al. Learning to detect a salient object[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(2): 353-367.
[11]	LI G B , XIE Y , LIN L ,et al. Instance-level salient object segmentation[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 247-256.
[12]	LIU T , DUAN H B , SHANG Y Y ,et al. Automatic salient object sequence rebuilding for video segment analysis[J]. Science China Information Sciences, 2018,61(1): 012205.
[13]	HOLZER S , POLLEFEYS M , ILIC S ,et al. Online learning of linear predictors for real-time tracking[C]// Proceedings of 2012 European Conference on Computer Vision. Heidelberg:Springer, 2012.
[14]	DANELLJAN M , H?GER G , KHAN F S ,et al. Accurate scale estimation for robust visual tracking[C]// Proceedings of the 2014 British Machine Vision Conference.[S.l.]: BMVA Press, 2014: 470-483.
[15]	CHEN D P , YUAN Z J , HUA G ,et al. Multi-timescale collaborative tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(1): 141-155.
[16]	CHANG H C , LAI S H , LU K R . A robust and efficient video stabilization algorithm[C]// Proceedings of 2004 IEEE International Conference on Multimedia and Expo. Piscataway:IEEE Press, 2004: 29-32.
[17]	BAY H , ESS A , TUYTELAARS T ,et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008,110(3): 346-359.
[18]	查冰, 张力, 艾海滨 . 不同特征提取算法对相机运动估计的适用性研究[J]. 测绘科学, 2018,43(3): 92-98.
	ZHA B , ZHANG L , AI H B . The applicability of different feature extraction algorithms to camera motion estimation[J]. Science of Surveying and Mapping, 2018,43(3): 92-98.
[19]	SHI J B , TOMASI . Good features to track[C]// Proceedings of 1994 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 1994: 593-600.
[20]	LUCAS B D , KANADE T . An iterative image registration technique with an application to stereo vision[C]// Proceedings of the 7th International Joint Conference on Artificial Intelligence. San Francisco:Morgan Kaufmann Publishers Inc, 1981: 674-679.
[21]	CAI J H , WALKER R . Robust motion estimation for camcorders mounted in mobile platforms[C]// Proceedings of 2008 Digital Image Computing:Techniques and Applications. Piscataway:IEEE Press, 2008: 491-497.
[22]	CHEN Q X , LIU T , SHANG Y Y ,et al. Salient object detection:integrate salient features in the deep learning framework[J]. IEEE Access, 2019,7: 152483-152492.
[23]	BOLME D S , BEVERIDGE J R , DRAPER B A ,et al. Visual object tracking using adaptive correlation filters[C]// Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2010: 2544-2550.
[24]	HARE S , GOLODETZ S , SAFFARI A ,et al. Struck:structured output tracking with kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38(10): 2096-2109.
[25]	BAKER S , MATTHEWS I . Lucas-Kanade 20 years on:a unifying framework[J]. International Journal of Computer Vision, 2004,56(3): 221-255.
[26]	BENHIMANE S , MALIS E . Real-time image-based tracking of planes using efficient second-order minimization[C]// Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2004: 943-948.
[27]	BENHIMANE S , MALIS E . Homography-based 2D visual tracking and servoing[J]. The International Journal of Robotics Research, 2007,26(7): 661-676.
[28]	LIU T , ZHENG N N , DING W ,et al. Video attention:learning to detect a salient object sequence[C]// Proceedings of 2008 19th International Conference on Pattern Recognition. Piscataway:IEEE Press, 2008: 1-4.
[29]	TIAN Y N , YANG G D , WANG Z ,et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019,157: 417-426.
[30]	TOMASI C , KANADE T . Detection and tracking of point features[J]. International Journal of Computer Vision, 1991,9(3): 137-154.
[31]	RAPHAEL E , KIEFER R , REISMAN P ,et al. Development of a camera-based forward collision alert system[J]. SAE International Journal of Passenger Cars-Mechanical Systems, 2011,4(1): 467-478.
[32]	刘世蔷, 黄影平, 胡兴 ,等. 基于自适应外点剔除与解耦算法的视觉里程计[J]. 信息与控制, 2019,48(2): 172-178.
	LIU S Q , HUANG Y P , HU X ,et al. Visual odometry based on adaptive outlier removal and decoupling algorithm[J]. Information and Control, 2019,48(2): 172-178.
[33]	CVI?I? I , ?ESI? J , MARKOVI? I ,et al. SOFT-SLAM:computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles[J]. Journal of Field Robotics, 2018,35(4): 578-595.
[34]	MUR-ARTAL R , TARDóS J D , . ORB-SLAM2:an open-source SLAM system for monocular,stereo,and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017,33(5): 1255-1262.
[35]	IN T , PAN J , CAO S Z ,et al. A general optimization-based framework for local odometry estimation with multiple sensors[J]. arXiv preprint,2019,arXiv:1901.03638.

Metrics

Recommended 0

No Suggested Reading articles found!

评价指标	BA	自适应BA	本文方法
TPR	66.7%	83.33%	85.58%
FPR	48%	28%	31%

类别	SSO	MSO	NSO
rot.error1/(°/m)	8.056 6	4.312 8	1.446 8
trans.error1	0.121 9%	0.072 0%	0.025 2%
rot.error2/(°/m)	4.882 2	3.289 9	1.451 7
trans.error2	0.099 4%	0.051 7%	0.025 3%

类别	平均旋转误差/(°/m)	平均平移误差
SOFT2	0.001 4	0.66%
ORB-SLAM2	0.002 7	1.15%
VINS-Fusion	0.003 3	1.09%
ERL	0.003 6	1.17%
本文方法	0.002 4	0.87%

视频序列	方法	ME	MAE	STD
刹车	本文方法	0.83%	0.17%	0.20%
	DSST	2.45%	1.08%	0.94%
慢速	本文方法	1.02%	0.37%	0.34%
	DSST	2.19%	1.33%	1.91%
静止	本文方法	1.91%	0.52%	0.58%
	DSST	3.56%	1.74%	2.61%

A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 35

Related Articles 10

Metrics

Recommended 0

[1]	Yuqian WANG, Rong DING. Automatic path planning program generation system based on swarm intelligence results [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 255-263.
[2]	Chenfeng GUO, Dongrui WU. A survey on canonical correlation analysis based multi-view learning [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 14-26.
[3]	Pu FENG, Wenjun WU, Jie LUO, Xin YU, Yongkai TIAN. Emergence measurement of robot swarm intelligence based on swarm entropy [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 65-74.
[4]	Junwei DUAN, Lincan XU, Yujuan QUAN, Long CHEN, C.L.Philip CHEN. Graph-regularized Bayesian broad learning system [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 109-117.
[5]	Han WANG, Yang YU, Yuan JIANG. A cooperative multi-agent reinforcement learning algorithm based on dynamic self-selection parameters sharing [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 75-83.
[6]	Kaishi YIN, Meng YANG, Xi GU, Zhicheng WANG. Light weight rotating object detector based on angle sensitive spatial attention mechanism [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 322-333.
[7]	Xi CHEN, Runmei LI, Jian WANG, Shuyun DONG. Fault discovery based on text information extraction for on-board equipment of CTCS [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 195-201.
[8]	Yi HU, Boyang QU, Jing LIANG, Jie WANG, Yanli WANG. A survey on evolutionary ensemble learning algorithm [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(1): 18-35.
[9]	Zhaoyang LIU, Chaoxu MU, Changyin SUN. An overview on algorithms and applications of deep reinforcement learning [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 314-326.
[10]	Bo ZHANG. Artificial intelligence is entering the post deep-learning era [J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(1): 4-6.