基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法

doi:10.11959/j.issn.2096-6652.202129

智能科学与技术学报 ›› 2021, Vol. 3 ›› Issue (3): 280-293.doi: 10.11959/j.issn.2096-6652.202129

• 专刊：目标智能检测与识别 • 上一篇下一篇

基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法

艾明欣¹, 刘铁¹, 王净¹, 丁佳丽¹, 袁泽剑², 尚媛园¹

¹ 首都师范大学信息工程学院，北京 100048
² 西安交通大学人工智能学院，陕西西安 710049

修回日期:2021-05-18 出版日期:2021-09-15 发布日期:2021-09-01
作者简介:艾明欣（1999− ），女，首都师范大学信息工程学院硕士生，主要研究方向为计算机视觉、模式识别等
刘铁（1981− ）男，博士，首都师范大学信息工程学院副教授，主要研究方向为计算机视觉、模式识别、多媒体计算、图像处理、大规模数据计算等
王净（1997− ），女，首都师范大学信息工程学院硕士生，主要研究方向为计算机视觉、模式识别等
丁佳丽（1996− ），女，首都师范大学信息工程学院硕士生，主要研究方向为计算机视觉、显著性物体检测等
袁泽剑（1971− ），男，博士，西安交通大学人工智能学院教授，主要研究方向为图像处理、模式识别、计算机视觉、机器学习、概率机器人等
尚媛园（1977− ），女，博士，首都师范大学信息工程学院教授，主要研究方向为图像处理、计算机视觉、嵌入式系统等
基金资助:
国家自然科学基金资助项目(61876112);国家自然科学基金资助项目(61976170);北京市自然科学基金资助项目(L201022)

A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation

Mingxin AI¹, Tie LIU¹, Jing WANG¹, Jiali DING¹, Zejian YUAN², Yuanyuan SHANG¹

¹ Information Engineering College, Capital Normal University, Beijing 100048, China
² College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an 710049, China

Revised:2021-05-18 Online:2021-09-15 Published:2021-09-01
Supported by:
The National Natural Science Foundation of China(61876112);The National Natural Science Foundation of China(61976170);The Natural Science Foundation of Beijing(L201022)

摘要/Abstract

摘要：

提出一种基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法。首先，针对车载相机自运动估计，通过视觉显著性计算方法检测并去除含有噪声的单目图像序列中的运动目标，同时考虑图像的纹理区域和平滑区域，利用加权显著图保留有用特征点，进而对车载相机进行鲁棒的自运动估计。其次，将前车距离转化为前车尺度估计问题，通过描述子匹配与李代数中正则化的强度匹配相结合的方法最小化损失函数，通过设计视觉注意力机制选择有纹理无遮挡的图像块，并对选定的图像块中的像素赋权以减轻被噪声破坏像素的影响，从而实现鲁棒、准确的尺度估计。最后，利用多个具有挑战性的数据集对所提方法进行分析验证。结果表明，单目相机自运动估计方法达到了基于立体相机方法的水平，前车尺度估计方法在充分发挥强鲁棒性优势的同时保证了预测精度。

关键词: 视觉显著性, 单目相机, 运动估计, 尺度估计

Abstract:

A method based on visual saliency for ego-motion estimation and scale estimation of the vehicle in front was proposed.Firstly, for the ego-motion estimation of vehicle-mounted camera, the visual saliency calculation method was used to detect and remove moving objects in the monocular image sequence containing noise.While considering the image texture and smooth region, the weighted saliency map was used to retain useful feature points, to improve the robustness of ego-motion estimation.Secondly, the distance of the vehicle in front was converted into a vehicle scale estimation, by integrating descriptor match and the strength of regularization match of the lie algebra to minimize loss function.The visual attention mechanism was used to get texture image block without shade, and the pixel in the image block weight to mitigate the effects of destroyed by noise pixel, so as to realize the robust and accurate scale estimation.Finally, several challenging datasets were used to analyze and verify the proposed method.The results show that the monocular camera ego-motion estimation method reaches the level of the stereo camera-based method, and the vehicle scale estimation method ensures the prediction accuracy while giving full play to the advantages of strong robustness.

Key words: visual saliency, monocular camera, motion estimation, scale estimation

中图分类号:

TP181

艾明欣, 刘铁, 王净, 等. 基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法[J]. 智能科学与技术学报, 2021, 3(3): 280-293.

Mingxin AI, Tie LIU, Jing WANG, et al. A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 280-293.

图/表 16

图1

图2

图3

图4

图5

表1

表2

图6

图7

表3

图8

图9

图10

表4

图11

图12

参考文献 35

[1]	JAEGLE A , PHILLIPS S , DANIILIDIS K . Fast,robust,continuous monocular egomotion computation[C]// Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA). Piscataway:IEEE Press, 2016: 773-780.
[2]	LOURAKIS M I A , ARGYROS A A . SBA:a software package for generic sparse bundle adjustment[J]. ACM Transactions on Mathematical Software, 2009,36(1): 1-30.
[3]	袁野, 和晓歌, 朱定坤 ,等. 视觉图像显著性检测综述[J]. 计算机科学, 2020,47(7): 84-91.
	YUAN Y , HE X G , ZHU D K ,et al. Survey of visual image saliency detection[J]. Computer Science, 2020,47(7): 84-91.
[4]	GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 3354-3361.
[5]	刘凯, 李浥东, 林伟鹏 . 车辆再识别技术综述[J]. 智能科学与技术学报, 2020,2(1): 10-25.
	LIU K , LI Y D , LIN W P . A survey on vehicle re-identification[J]. Chinese Journal of Intelligent Science and Technology, 2020,2(1): 10-25.
[6]	熊刚, 董西松, 朱凤华 ,等. 城市交通大数据技术及智能应用系统[J]. 大数据, 2015,1(4): 81-96.
	XIONG G , DONG X S , ZHU F H ,et al. Big data technologies and intelligent application system for urban transportation[J]. Big Data Research, 2015,1(4): 81-96.
[7]	KIM G , CHO J S . Vision-based vehicle detection and inter-vehicle distance estimation[C]// Proceedings of 2012 12th International Conference on Control,Automation and Systems. Piscataway:IEEE Press, 2012: 625-629.
[8]	MUKHTAR A , XIA L K , TANG T B . Vehicle detection techniques for collision avoidance systems:a review[J]. IEEE Transactions on Intelligent Transportation Systems, 2015,16(5): 2318-2338.
[9]	DAGAN E , MANO O , STEIN G P ,et al. Forward collision warning with a single camera[C]// Proceedings of 2004 IEEE Intelligent Vehicles Symposium. Piscataway:IEEE Press, 2004: 37-42.
[10]	LIU T , YUAN Z J , SUN J ,et al. Learning to detect a salient object[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(2): 353-367.
[11]	LI G B , XIE Y , LIN L ,et al. Instance-level salient object segmentation[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 247-256.
[12]	LIU T , DUAN H B , SHANG Y Y ,et al. Automatic salient object sequence rebuilding for video segment analysis[J]. Science China Information Sciences, 2018,61(1): 012205.
[13]	HOLZER S , POLLEFEYS M , ILIC S ,et al. Online learning of linear predictors for real-time tracking[C]// Proceedings of 2012 European Conference on Computer Vision. Heidelberg:Springer, 2012.
[14]	DANELLJAN M , H?GER G , KHAN F S ,et al. Accurate scale estimation for robust visual tracking[C]// Proceedings of the 2014 British Machine Vision Conference.[S.l.]: BMVA Press, 2014: 470-483.
[15]	CHEN D P , YUAN Z J , HUA G ,et al. Multi-timescale collaborative tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(1): 141-155.
[16]	CHANG H C , LAI S H , LU K R . A robust and efficient video stabilization algorithm[C]// Proceedings of 2004 IEEE International Conference on Multimedia and Expo. Piscataway:IEEE Press, 2004: 29-32.
[17]	BAY H , ESS A , TUYTELAARS T ,et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008,110(3): 346-359.
[18]	查冰, 张力, 艾海滨 . 不同特征提取算法对相机运动估计的适用性研究[J]. 测绘科学, 2018,43(3): 92-98.
	ZHA B , ZHANG L , AI H B . The applicability of different feature extraction algorithms to camera motion estimation[J]. Science of Surveying and Mapping, 2018,43(3): 92-98.
[19]	SHI J B , TOMASI . Good features to track[C]// Proceedings of 1994 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 1994: 593-600.
[20]	LUCAS B D , KANADE T . An iterative image registration technique with an application to stereo vision[C]// Proceedings of the 7th International Joint Conference on Artificial Intelligence. San Francisco:Morgan Kaufmann Publishers Inc, 1981: 674-679.
[21]	CAI J H , WALKER R . Robust motion estimation for camcorders mounted in mobile platforms[C]// Proceedings of 2008 Digital Image Computing:Techniques and Applications. Piscataway:IEEE Press, 2008: 491-497.
[22]	CHEN Q X , LIU T , SHANG Y Y ,et al. Salient object detection:integrate salient features in the deep learning framework[J]. IEEE Access, 2019,7: 152483-152492.
[23]	BOLME D S , BEVERIDGE J R , DRAPER B A ,et al. Visual object tracking using adaptive correlation filters[C]// Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2010: 2544-2550.
[24]	HARE S , GOLODETZ S , SAFFARI A ,et al. Struck:structured output tracking with kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38(10): 2096-2109.
[25]	BAKER S , MATTHEWS I . Lucas-Kanade 20 years on:a unifying framework[J]. International Journal of Computer Vision, 2004,56(3): 221-255.
[26]	BENHIMANE S , MALIS E . Real-time image-based tracking of planes using efficient second-order minimization[C]// Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2004: 943-948.
[27]	BENHIMANE S , MALIS E . Homography-based 2D visual tracking and servoing[J]. The International Journal of Robotics Research, 2007,26(7): 661-676.
[28]	LIU T , ZHENG N N , DING W ,et al. Video attention:learning to detect a salient object sequence[C]// Proceedings of 2008 19th International Conference on Pattern Recognition. Piscataway:IEEE Press, 2008: 1-4.
[29]	TIAN Y N , YANG G D , WANG Z ,et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019,157: 417-426.
[30]	TOMASI C , KANADE T . Detection and tracking of point features[J]. International Journal of Computer Vision, 1991,9(3): 137-154.
[31]	RAPHAEL E , KIEFER R , REISMAN P ,et al. Development of a camera-based forward collision alert system[J]. SAE International Journal of Passenger Cars-Mechanical Systems, 2011,4(1): 467-478.
[32]	刘世蔷, 黄影平, 胡兴 ,等. 基于自适应外点剔除与解耦算法的视觉里程计[J]. 信息与控制, 2019,48(2): 172-178.
	LIU S Q , HUANG Y P , HU X ,et al. Visual odometry based on adaptive outlier removal and decoupling algorithm[J]. Information and Control, 2019,48(2): 172-178.
[33]	CVI?I? I , ?ESI? J , MARKOVI? I ,et al. SOFT-SLAM:computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles[J]. Journal of Field Robotics, 2018,35(4): 578-595.
[34]	MUR-ARTAL R , TARDóS J D , . ORB-SLAM2:an open-source SLAM system for monocular,stereo,and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017,33(5): 1255-1262.
[35]	IN T , PAN J , CAO S Z ,et al. A general optimization-based framework for local odometry estimation with multiple sensors[J]. arXiv preprint,2019,arXiv:1901.03638.

评价指标	BA	自适应BA	本文方法
TPR	66.7%	83.33%	85.58%
FPR	48%	28%	31%

类别	SSO	MSO	NSO
rot.error1/(°/m)	8.056 6	4.312 8	1.446 8
trans.error1	0.121 9%	0.072 0%	0.025 2%
rot.error2/(°/m)	4.882 2	3.289 9	1.451 7
trans.error2	0.099 4%	0.051 7%	0.025 3%

类别	平均旋转误差/(°/m)	平均平移误差
SOFT2	0.001 4	0.66%
ORB-SLAM2	0.002 7	1.15%
VINS-Fusion	0.003 3	1.09%
ERL	0.003 6	1.17%
本文方法	0.002 4	0.87%

视频序列	方法	ME	MAE	STD
刹车	本文方法	0.83%	0.17%	0.20%
	DSST	2.45%	1.08%	0.94%
慢速	本文方法	1.02%	0.37%	0.34%
	DSST	2.19%	1.33%	1.91%
静止	本文方法	1.91%	0.52%	0.58%
	DSST	3.56%	1.74%	2.61%

基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法

A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 35

相关文章 10

Metrics

推荐阅读 0

[1]	王雨倩, 丁嵘. 基于群体智能成果的路径规划程序自动生成系统[J]. 智能科学与技术学报, 2022, 4(2): 255-263.
[2]	郭陈凤, 伍冬睿. 基于典型相关分析的多视图学习方法综述[J]. 智能科学与技术学报, 2022, 4(1): 14-26.
[3]	冯埔, 吴文峻, 罗杰, 于鑫, 田雍恺. 基于群体熵的机器人群体智能汇聚度量[J]. 智能科学与技术学报, 2022, 4(1): 65-74.
[4]	段俊伟, 许林灿, 全渝娟, 陈龙, 陈俊龙. 基于图正则化的贝叶斯宽度学习系统[J]. 智能科学与技术学报, 2022, 4(1): 109-117.
[5]	王涵, 俞扬, 姜远. 基于动态自选择参数共享的合作多智能体强化学习算法[J]. 智能科学与技术学报, 2022, 4(1): 75-83.
[6]	尹开石, 杨萌, 顾曦, 王志成. 基于角度敏感的空间注意力机制的轻量型旋转目标检测器[J]. 智能科学与技术学报, 2021, 3(3): 322-333.
[7]	陈曦, 李润梅, 王剑, 董舒韵. 基于文本信息抽取的列控车载设备故障发现[J]. 智能科学与技术学报, 2021, 3(2): 195-201.
[8]	胡毅, 瞿博阳, 梁静, 王杰, 王艳丽. 进化集成学习算法综述[J]. 智能科学与技术学报, 2021, 3(1): 18-35.
[9]	刘朝阳, 穆朝絮, 孙长银. 深度强化学习算法与应用研究现状综述[J]. 智能科学与技术学报, 2020, 2(4): 314-326.
[10]	张钹. 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019, 1(1): 4-6.