结合粒子滤波及度量学习的目标跟踪方法

doi:10.11959/j.issn.1000-436x.2021087

通信学报 ›› 2021, Vol. 42 ›› Issue (5): 98-110.doi: 10.11959/j.issn.1000-436x.2021087

结合粒子滤波及度量学习的目标跟踪方法

王洪雁¹^,²^,³, 张莉彬², 陈国强⁴, 汪祖民², 管志远⁵

¹ 浙江理工大学信息学院，浙江杭州 310018
² 大连大学信息工程学院，辽宁大连 116622
³ 五邑大学智能制造学部，广东江门 529020
⁴ 河南大学计算机与信息工程学院，河南开封 475004
⁵ 杭州电子科技大学电子信息学院，浙江杭州 310018

修回日期:2021-02-04 出版日期:2021-05-25 发布日期:2021-05-01
作者简介:王洪雁（1979- ），男，河南南阳人，博士，浙江理工大学特聘教授、硕士生导师，主要研究方向为阵列信号处理、机器视觉等
张莉彬（1995- ），女，山西吕梁人，大连大学硕士生，主要研究方向为图像处理、视觉追踪等
陈国强（1977- ），男，河南开封人，博士，河南大学副教授、硕士生导师，主要研究方向为机器视觉、优化理论等
汪祖民（1975- ），男，河南信阳人，博士，大连大学教授、硕士生导师，主要研究方向为信号处理、机器学习等
管志远（1997- ），男，河南鹤壁人，杭州电子科技大学硕士生，主要研究方向为信号处理、机器学习等
基金资助:
国家自然科学基金资助项目(61301258);国家自然科学基金资助项目(61871164);浙江省自然科学基金资助项目(LZ21F010002);中国博士后科学基金资助项目(2016M590218);河南省科技攻关计划基金资助项目(162102210168);浙江理工大学科研启动基金资助项目(21032098-Y)

Approach of target tracking combining particle filter and metric learning

Hongyan WANG¹^,²^,³, Libin ZHANG², Guoqiang CHEN⁴, Zumin WANG², Zhiyuan GUAN⁵

¹ School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018, China
² College of Information Engineering, Dalian University, Dalian 116622, China
³ Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529020, China
⁴ School of Computer and Information Engineering, Henan University, Kaifeng 475004, China
⁵ School of Electronic Information, Hangzhou Dianzi University, Hangzhou 310018, China

Revised:2021-02-04 Online:2021-05-25 Published:2021-05-01
Supported by:
The National Natural Science Foundation of China(61301258);The National Natural Science Foundation of China(61871164);The Natural Science Foundation of Zhejiang Province(LZ21F010002);China Postdoctoral Science Foundation(2016M590218);Science and Technology Re-search Plan of Henan Province(162102210168);Startup Foundation Project of Zhejiang Sci-Tech University(21032098-Y)

摘要/Abstract

摘要：

针对复杂环境导致目标跟踪性能显著下降的问题，提出基于粒子滤波与度量学习的目标跟踪方法。所提方法首先离线训练可高效获取目标特征的卷积神经网络（CNN）；其次，基于核回归度量学习（MLKR）方法构建最小化预测误差的距离度量矩阵优化模型，并利用梯度下降法求解所得模型以获得候选目标最优解；再次，基于最优候选预测值计算重构误差以构建目标观测模型；最后，引入长短时稳定更新策略并基于粒子滤波跟踪框架实现有效跟踪。实验结果表明，复杂环境下所提方法具有较高跟踪精度及较好稳健性。

关键词: 目标跟踪, 粒子滤波, 卷积神经网络, 度量学习, 稀疏表示

Abstract:

Focusing on the issue of the significant degradation of target tracking performance caused by adverse factors in complex environment, a target tracking method based on particle filtering and metric learning was proposed.First of all, a convolutional neural network (CNN) was offline-trained via the proposed method to effectively obtain the target characteristics.After that, the distance measurement matrix optimization model to minimize the prediction error could be constructed on the basis of the metric learning for kernel regression (MLKR) method, and the resultant model could be handled via using the gradient descent approach to obtain the optimal solution of the candidate target.Moreover, based on the predicted value of the optimal candidate target, the reconstruction error was calculated to construct the target observation model.Finally, a long-short-term update strategy was introduced to achieve the effective target tracking under the particle filter tracking framework.The experiment results show that the proposed method has higher tracking accuracy and better robustness in complex environments.

Key words: target tracking, particle filter, convolutional neural network, metric learning, sparse representation

中图分类号:

TP391

王洪雁, 张莉彬, 陈国强, 汪祖民, 管志远. 结合粒子滤波及度量学习的目标跟踪方法[J]. 通信学报, 2021, 42(5): 98-110.

Hongyan WANG, Libin ZHANG, Guoqiang CHEN, Zumin WANG, Zhiyuan GUAN. Approach of target tracking combining particle filter and metric learning[J]. Journal on Communications, 2021, 42(5): 98-110.

图/表 12

图1

图2

表1

图3

表2

不同跟踪方法在各测试序列的平均跟踪重叠率"

测试序列	所提方法	CNN-PF	Struck	CT	TLD	DFT	BACF
MotorRolling	$0 . 71$	0.48	0.19	0.21	0.27	0.16	$\underline{0.67}$
Jogging	$0 . 81$	$\underline{0.75}$	0.19	0.11	0.66	0.21	0.61
Boy	$0 . 82$	$\underline{0.76}$	0.65	0.39	0.61	0.67	0.56
Skating1	$0 . 48$	0.35	$\underline{0.43}$	0.04	0.31	0.06	0.24
Matrix	$0 . 65$	$\underline{0.45}$	0.01	0.02	0.16	0.02	0.37
Bird2	$0 . 77$	0.57	0.37	0.54	$\underline{0.61}$	0.59	0.52
Tiger2	$0 . 77$	0.65	0.18	0.13	0.24	0.63	$\underline{0.74}$
Basketball	$0 . 72$	0.51	0.35	$\underline{0.57}$	0.55	0.54	0.55
Singer1	$0 . 68$	0.33	0.39	0.45	$\underline{0.51}$	0.38	0.47
Singer2	$0 . 63$	$\underline{0.57}$	0.35	0.29	0.48	0.39	0.54
平均值	$0 . 70$	$\underline{0.54}$	0.31	0.28	0.44	0.37	0.52

表2

表3

不同跟踪方法在各测试序列的平均中心位置误差"

测试序列	所提方法	CNN-PF	Struck	CT	TLD	DFT	BACF
MotorRolling	$15 . 95$	$\underline{26.56}$	148.55	132.75	79.81	164.54	148.55
Jogging	$3 . 27$	15.14	107.93	139.63	13.53	33.76	$\underline{5.33}$
Boy	$1 . 57$	1.96	3.36	34.03	3.60	2.42	1.58
Skating1	$\underline{13.68}$	$9 . 33$	65.58	180.11	125.17	178.15	140.10
Matrix	$10 . 87$	$\underline{24.41}$	221.08	85.94	57.23	206.55	113.30
Bird2	$10 . 11$	44.17	65.09	$\underline{17.91}$	37.61	47.78	21.59
Tiger2	$11 . 56$	$\underline{14.06}$	66.82	70.22	53.32	14.25	21.52
Basketball	$15 . 51$	$\underline{16.02}$	80.93	121.33	72.45	181.29	49.60
Singer1	$19 . 73$	55.21	71.92	40.20	$\underline{25.53}$	49.53	69.04
Singer2	$\underline{12.75}$	$9 . 82$	66.39	115.23	47.28	51.86	57.57
平均值	$11 . 50$	$\underline{21.67}$	89.77	93.74	51.56	92.49	62.81

表3

图4

图5

图6

图7

表4

表5

不同测试序列中各跟踪方法的FPS"

测试序列	所提方法	CNN-PF	Struck	CT	TLD	DFT	BACF
MotorRolling	19.21	7.96	12.14	$46 . 53$	31.87	2.82	18.27
Jogging	4.13	8.92	12.16	$58 . 33$	17.03	8.27	19.86
Boy	16.39	8.38	12.81	$26 . 43$	16.05	5.38	22.46
Skating1	8.71	8.54	9.73	$37 . 69$	24.71	4.38	18.81
Matrix	9.23	10.12	8.24	$30 . 78$	23.17	6.35	21.01
Bird2	32.38	19.31	15.77	$42 . 12$	27.32	5.56	23.16
Tiger2	28.17	13.79	6.32	$43 . 86$	9.37	4.60	21.52
Basketball	19.65	10.44	5.59	$41 . 34$	21.11	8.65	28.37
Singer1	13.13	9.37	11.99	$38 . 83$	12.26	1.81	16.25
Singer2	9.91	10.62	6.58	$40 . 93$	27.63	3.28	16.66
平均	16.09	10.74	10.13	$40 . 68$	21.05	5.11	20.63

表5

参考文献 31

[1]	ZHANG N N , WU C X , WU Y ,et al. An improved target tracking algorithm and its application in intelligent video surveillance system[J]. Multimedia Tools and Applications, 2020,79(23/24): 15965-15983.
[2]	FRUHWIRTH-REISINGER C , KRISPEL G , POSSEGGER H ,et al. Towards data-driven multi-target tracking for autonomous driving[C]// Computer Vision Winter Workshop.[S.n.:s.l.], 2020.
[3]	YUAN M Z . An analysis model of sports human body based on computer vision tracking technology[J]. DEStech Transactions on Social Science,Education and Human Science, 2017.
[4]	BAISA N L . Robust online multi-target visual tracking using a HISP filter with discriminative deep appearance learning[J]. Journal of Visual Communication and Image Representation, 2021,77: 102952.
[5]	王铮, 赵晓, 佘宏杰 ,等. 基于双目视觉的AGV障碍物检测与避障[J]. 计算机集成制造系统, 2018,24(2): 400-409.
	WANG Z , ZHAO X , SHE H J ,et al. Obstacle detection and obstacle avoidance of AGV based on binocular vision[J]. Computer Integrated Manufacturing Systems, 2018,24(2): 400-409.
[6]	PAN Z , LIU S , FU W N . A review of visual moving target tracking[J]. Multimedia Tools and Applications, 2017,76(16): 16989-17018.
[7]	KRISTAN M , LEONARDIS A , MATAS J ,et al. The visual object tracking vot2017 challenge results[C]// 2017 the IEEE International Conference on Computer Vision Workshop. Piscataway:IEEE Press, 2017: 1949-1972.
[8]	WANG N , YEUNG D Y . Learning a deep compact image representation for visual tracking[C]// Advances in Neural Information Processing Systems. Massachusetts:MIT Press, 2013: 809-817.
[9]	WANG L J , OUYANG W L , WANG X G ,et al. Visual tracking with fully convolutional networks[C]// 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2015: 3119-3127.
[10]	ZHANG K H , LIU Q S , WU Y ,et al. Robust visual tracking via convolutional networks without training[C]// IEEE Transactions on Image Processing. Piscataway:IEEE Press, 2016: 1779-1792.
[11]	MOZHDEHI R J , MEDEIROS H . Deep convolutional particle filter for visual tracking[C]// 2017 IEEE International Conference on Image Processing. Piscataway:IEEE Press,, 2017: 3650-3654.
[12]	HU J L , LU J W , TAN Y P . Deep metric learning for visual tracking[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016,26(11): 2056-2068.
[13]	WEINBERGER K Q , TESAURO G . Metric learning for kernel regression[C]// 2007 Artificial Intelligence and Statistics.[S.n.:s.l.], 2007: 612-619.
[14]	HUANG R Q , SUN S L . Kernel regression with sparse metric learning[J]. Journal of Intelligent ＆ Fuzzy Systems, 2013,24(4): 775-787.
[15]	NADARAYA E A . On estimating regression[J]. Theory of Probability＆ Its Applications, 1964,9(1): 141-142.
[16]	GUTIERREZ E D , LEVY R , BERGEN B . Finding non-arbitrary form-meaning systematicity using string-metric learning for kernel regression[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.n.:s.l.], 2016: 2379-2388.
[17]	WANG N , YEUNG D . Learning a deep compact image representation for visual tracking[C]// Neural Information Processing Systems.[S.n.:s.l.], 2013: 809-817.
[18]	POLSON N , SOKOLOV V . Bayesian particle tracking of traffic flows[J]. IEEE Transactions on Intelligent Transportation Systems, 2018,19(2): 345-356.
[19]	ROSS D A , LIM J , LIN R S ,et al. Incremental learning for robust visual tracking[J]. International Journal of Computer Vision, 2008,77(1): 125-141.
[20]	XIE S C , ZHANG X H , CAI J . Video crowd detection and abnormal behavior model detection based on machine learning method[J]. Neural Computing and Applications, 2019,31(1): 175-184.
[21]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems.[S.n.:s.l.], 2012: 1097-1105.
[22]	HE Z Y , YI S Y , CHEUNG Y M ,et al. Robust object tracking via key patch sparse representation[J]. IEEE Transactions on Cybernetics, 2017,47(2): 354-364.
[23]	王洪雁, 邱贺磊, 郑佳 ,等. 光照变化下基于逆向稀疏表示的视觉跟踪方法[J]. 电子与信息学报, 2019,41(3): 632-639.
	WANG H Y , QIU H L , ZHENG J ,et al. Visual tracking method based on reverse sparse representation under illumination variation[J]. Journal of Electronics ＆ Information Technology, 2019,41(3): 632-639.
[24]	WU Y , LIM J , YANG M H . Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(9): 1834-1848.
[25]	KRISTAN M , LEONARDIS A , MATAS J ,et al. The sixth visual object tracking VOT2018 challenge results[J]// European Conference on Computer Vision.Berlin:Springer, 2018: 3-53.
[26]	李位星, 马维亮, 田卉 ,等. 基于CNN的粒子滤波目标跟踪算法研究[J]. 北京理工大学学报, 2018,38(12): 1256-1262.
	LI W X , MA W L , TIAN H ,et al. Particle filter for object tracking based on CNN feature[J]. Transactions of Beijing Institute of Technology, 2018,38(12): 1256-1262.
[27]	HARE S , GOLODETZ S , SAFFARI A ,et al. Struck:structured output tracking with kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38(10): 2096-2109.
[28]	ZHANG K H , ZHANG L , YANG M H ,et al. Real-time compressive tracking[C]// European Conference on Computer Vision. Berlin:Springer, 2012: 864-877.
[29]	KALAL Z , MATAS J , MIKOLAJCZYK K . P-N learning:Bootstrapping binary classifiers by structural constraints[C]// 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2010: 49-56.
[30]	SEVILLA-LARA L , LEARNED-MILLER E . Distribution fields for tracking[C]// 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 1910-1917.
[31]	GALOOGAHI H K , FAGG A , LUCEY S . Learning background-aware correlation filters for visual tracking[C]// 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 1144-1152.

测试序列	光照变化	比例变化	运动模糊	旋转	遮挡	快速运动	背景杂波
MotorRolling	√	√	√	×	×	×	√
Jogging	×	×	×	×	√	×	×
Boy	×	√	×	√	×	√	×
Skating1	√	√	√	√	√	√	√
Matrix	×	√	×	√	×	√	√
Bird2	×	×	√	√	×	√	√
Tiger2	×	×	√	×	√	√	√
Basketball	×	×	√	×	√	√	√
Singer1	√	√	√	×	×	×	√
Singer2	√	×	√	×	×	√	√

独立仿真项	测试序列
独立仿真项	Boy	MotorRolling
CNN+粒子滤波+长短时更新	0.76	0.64
CNN+MLKR +长短时更新	0.77	0.67
CNN+粒子滤波+MLKR	0.80	0.69
CNN+MLKR+粒子滤波+长短时更新	0.82	0.71

结合粒子滤波及度量学习的目标跟踪方法

Approach of target tracking combining particle filter and metric learning

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 31

相关文章 15

Metrics

推荐阅读 0

[1]	曾浩, 母王强, 杨顺平. 高机动目标跟踪ATPM-IMM算法[J]. 通信学报, 2022, 43(7): 93-101.
[2]	李昂, 陈建新, 魏昕, 周亮. 面向6G的跨模态信号重建技术[J]. 通信学报, 2022, 43(6): 28-40.
[3]	王晓丹, 李京泰, 宋亚飞. DDAC：面向卷积神经网络图像隐写分析模型的特征提取方法[J]. 通信学报, 2022, 43(5): 68-81.
[4]	廖育荣, 王海宁, 林存宝, 李阳, 方宇强, 倪淑燕. 基于深度学习的光学遥感图像目标检测研究进展[J]. 通信学报, 2022, 43(5): 190-203.
[5]	张帆, 黄赟, 方子茁, 郭威. 卷积神经网络的损失最小训练后参数量化方法[J]. 通信学报, 2022, 43(4): 114-122.
[6]	朱政宇, 侯庚旺, 黄崇文, 孙钢灿, 郝万明, 梁静. 基于并行CNN的RIS辅助D2D保密通信系统资源分配算法[J]. 通信学报, 2022, 43(3): 172-179.
[7]	陈志旺, 张忠新, 宋娟, 雷海鹏, 彭勇. 在线目标分类及自适应模板更新的孪生网络跟踪算法[J]. 通信学报, 2021, 42(8): 151-163.
[8]	安泽亮, 张天骐, 马宝泽, 邓盼, 徐雨晴. 基于一维CNN的多入多出OSTBC信号协作调制识别[J]. 通信学报, 2021, 42(7): 84-94.
[9]	司念文, 张文林, 屈丹, 常禾雨, 李盛祥, 牛铜. 基于对抗补丁的可泛化的Grad-CAM攻击方法[J]. 通信学报, 2021, 42(3): 23-35.
[10]	高红民, 曹雪莹, 陈忠昊, 花再军, 李臣明, 陈月. 基于多尺度近端特征拼接网络的高光谱图像分类方法[J]. 通信学报, 2021, 42(2): 92-102.
[11]	程旭, 王莹莹, 张年杰, 付章杰, 陈北京, 赵国英. 基于空间感知的多级损失目标跟踪对抗攻击方法[J]. 通信学报, 2021, 42(11): 242-254.
[12]	杨军,党吉圣. 基于上下文注意力CNN的三维点云语义分割[J]. 通信学报, 2020, 41(7): 195-203.
[13]	高红民,曹雪莹,杨耀,花再军,李臣明. 基于CNN的双边融合网络在高光谱图像分类中的应用[J]. 通信学报, 2020, 41(11): 132-140.
[14]	张猛,孙昊良,杨鹏. 基于改进卷积神经网络识别DNS隐蔽信道[J]. 通信学报, 2020, 41(1): 169-179.
[15]	裴家正,黄勇,董云龙,陈小龙. 改进的SMC-CBMeMBer前向后向平滑检测前跟踪算法[J]. 通信学报, 2019, 40(8): 102-113.