在线目标分类及自适应模板更新的孪生网络跟踪算法

doi:10.11959/j.issn.1000-436x.2021127

通信学报 ›› 2021, Vol. 42 ›› Issue (8): 151-163.doi: 10.11959/j.issn.1000-436x.2021127

在线目标分类及自适应模板更新的孪生网络跟踪算法

陈志旺¹, 张忠新¹, 宋娟², 雷海鹏¹, 彭勇²

¹ 燕山大学工业计算机控制工程河北省重点实验室，河北秦皇岛 066004
² 国网黑龙江省电力有限公司佳木斯供电公司，黑龙江佳木斯 154002
³ 燕山大学电气工程学院，河北秦皇岛 066004

修回日期:2021-04-10 出版日期:2021-08-25 发布日期:2021-08-01
作者简介:陈志旺（1978- ），男，河北武清人，博士，燕山大学副教授、硕士生导师，主要研究方向为多旋翼飞行控制、目标跟踪等
张忠新（1996- ），男，山东聊城人，燕山大学硕士生，主要研究方向为目标跟踪
宋娟（1978- ），女，黑龙江佳木斯人，国网黑龙江省电力有限公司高级工程师，主要研究方向为发电厂智能控制
雷海鹏（1997- ），男，河北张家口人，燕山大学硕士生，主要研究方向为目标跟踪
彭勇（1963- ），男，河北唐山人，博士，燕山大学教授、博士生导师，主要研究方向为特种机器人与人工智能
基金资助:
国家自然科学基金资助项目(61573305);河北省自然科学基金资助项目(F2019203511)

Tracking algorithm of Siamese network based on online target classification and adaptive template update

Zhiwang CHEN¹, Zhongxin ZHANG¹, Juan SONG², Haipeng LEI¹, Yong PENG²

¹ Key Lab of Industrial Computer Control Engineering of Hebei Province, Yanshan University, Qinhuangdao 066004, China
² Jiamusi Electric Power Company, State Grid Heilongjiang Electric Power Co., Ltd., Jiamusi 154002, China
³ School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China

Revised:2021-04-10 Online:2021-08-25 Published:2021-08-01
Supported by:
The National Natural Science Foundation of China(61573305);The Natural Science Foundation of Hebei Province(F2019203511)

摘要/Abstract

摘要：

针对孪生网络跟踪算法在离线训练阶段学习被跟踪目标和其他对象的嵌入式特征，而这些特征缺少特定于目标的上下文信息，使跟踪算法的稳健性较差的问题，以SiamRPN++作为基准算法，提出了在线目标分类及自适应模板更新的孪生网络跟踪算法。首先，在离线训练阶段设计了互相关特征图监督模块，以学习更具判别力的嵌入式特征；其次，在线跟踪阶段设计了包含注意力机制的在线目标分类模块，在该模块中使用在线滤波器更新策略滤除背景噪声干扰；最后，设计了一种自适应模板更新模块，使用UpdateNet更新目标模板信息。在VOT2018、VOT2019这2个标准数据集上的实验结果验证了所提算法的有效性，相比基准算法SiamRPN++分别带来13.5%和18.2%（EAO）的性能提升。

关键词: 机器视觉, 目标跟踪, 孪生网络, 目标分类, 自适应模板更新

Abstract:

Aiming at the problem that tracking algorithm of Siamese network learned the embedded features of the tracked target and the object in the offline training stage, and these embedded features often lacked the target-specific context information, which made these tracking algorithms less robust, a tracking algorithm of the Siamese network based on online target classification and adaptive template update was proposed, which used SiamRPN++ as the baseline algorithm.Firstly, a cross-correlation feature map supervision module for classification was designed in the offline training phase to learn more discriminative embedded features.Secondly, an online target classification module that included an attention mechanism in the online tracking phase was designed, and the online update filter strategy in the module was used to filter out the background noise.Finally, an adaptive template update module was designed to update the target template information using the UpdateNet.The results of experiments on VOT2018 and VOT2019 datasets verify the effectiveness of the proposed algorithm, which brings 13.5% and 18.2% (EAO) improvement respectively compared with the baseline algorithm SiamRPN++.

Key words: machine vision, object tracking, Siamese network, object classification, adaptive template update

中图分类号:

TP391.4

陈志旺, 张忠新, 宋娟, 雷海鹏, 彭勇. 在线目标分类及自适应模板更新的孪生网络跟踪算法[J]. 通信学报, 2021, 42(8): 151-163.

Zhiwang CHEN, Zhongxin ZHANG, Juan SONG, Haipeng LEI, Yong PENG. Tracking algorithm of Siamese network based on online target classification and adaptive template update[J]. Journal on Communications, 2021, 42(8): 151-163.

图/表 11

图1

图2

图3

图4

图5

表1

表2

表3

表4

表5

表6

参考文献 24

[22]	BHAT G , DANELLJAN M , VAN GOOL L ,et al. Learning discriminative model prediction for tracking[C]// 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 6181-6190.
[23]	XU Y D , WANG Z Y , LI Z X ,et al. SiamFC++:towards robust and accurate visual tracking with target estimation guidelines[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(7): 12549-12556.
[1]	BOLME D S , BEVERIDGE J R , DRAPER B A ,et al. Visual object tracking using adaptive correlation filters[C]// 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2010: 2544-2550.
[2]	HENRIQUES J F , CASEIRO R , MARTINS P ,et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(3): 583-596.
[3]	DANELLJAN M , H?GER G ,, KHAN F S , et al . Learning spatially regularized correlation filters for visual tracking[C]// 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2015: 4310-4318.
[4]	DANELLJAN M , BHAT G , KHAN F S ,et al. ECO:efficient convolution operators for tracking[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 6931-6939.
[5]	TAO R , GAVVES E , SMEULDERS A W M . Siamese instance search for tracking[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 1420-1429.
[6]	BERTINETTO L , VALMADRE J , HENRIQUES J F ,et al. Fully-convolutional Siamese networks for object tracking[M]. Berlin: Springer, 2016.
[7]	LI B , YAN J J , WU W ,et al. High performance visual tracking with Siamese region proposal network[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 8971-8980.
[8]	ZHU Z , WANG Q , LI B ,et al. Distractor-aware Siamese networks for visual object tracking[M]. Berlin: Springer, 2018.
[9]	ZHANG Z P , PENG H W . Deeper and wider Siamese networks for real-time visual tracking[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 4586-4595.
[10]	LI B , WU W , WANG Q ,et al. SiamRPN++:evolution of Siamese visual tracking with very deep networks[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 4277-4286.
[11]	WANG Q , ZHANG L , BERTINETTO L ,et al. Fast online object tracking and segmentation:a unifying approach[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 1328-1338.
[12]	陈志旺, 张忠新, 宋娟 ,等. 基于目标感知特征筛选的孪生网络跟踪算法[J]. 光学学报, 2020,40(9): 0915003.
	CHEN Z W , ZHANG Z X , SONG J ,et al. Tracking algorithm for Siamese network based on target-aware feature selection[J]. Acta Optica Sinica, 2020,40(9): 0915003.
[13]	LIU W Y , WEN Y D , YU Z D ,et al. Large-margin softmax loss for convolutional neural networks[J]. arXiv Preprint,arXiv:1612.02295, 2016.
[14]	LI X , MA C , WU B Y ,et al. Target-aware deep tracking[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 1369-1378.
[15]	ZHOU J H , WANG P , SUN H Y . Discriminative and robust online learning for Siamese visual tracking[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(7): 13017-13024.
[16]	DANELLJAN M , BHAT G , KHAN F S ,et al. ATOM:accurate tracking by overlap maximization[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 4655-4664.
[17]	MATEJ K , JIRI M , ALES L ,et al. The seventh visual object tracking VOT2019 challenge results[C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop. Piscataway:IEEE Press, 2019: 2206-2241.
[18]	ZHANG L C , GONZALEZ-GARCIA A ,, WEIJER J V D , et al . Learning the model update for Siamese trackers[C]// 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 4009-4018.
[19]	KRISTAN M , LEONARDIS A , MATAS J ,et al. The sixth visual object tracking VOT2018 challenge results[C]// Proceedings of the European Conference on Computer Vision. Berlin:Springer, 2018: 3-53.
[20]	FAN H , LIN L T , YANG F ,et al. LaSOT:a high-quality benchmark for large-scale single object tracking[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 5369-5378.
[21]	CHEN Z D , ZHONG B N , LI G R ,et al. Siamese box adaptive network for visual tracking[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 6667-6676.
[24]	VOIGTLAENDER P , LUITEN J , TORR P H S ,et al. Siam R-CNN:visual tracking by Re-detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 6577-6587.

方案	A	R	EAO	LN/次	V_FPS/(帧.s^-1)
SiamRPN++	0.601	0.234	0.415	50	35
+ CS module	0.583	0.197	0.432	42	62

方案	A	R	EAO	LN/次	V_FPS/(帧.s^-1)
SiamRPN++	0.601	0.234	0.415	50	35
+OC module	0.611	0.150	0.417	32	28
+ OC module + CS module	0.591	0.140	0.463	30	41

方案	online update			A	R	LN/次	EAO
方案	cmp	attn	flt	A	R	LN/次	EAO
SiamRPN++	×	×	×	0.601	0.234	50	0.415
+ CS module +OC module	√	√	√	0.591	0.140	30	0.463
+CS module +OC module	×	√	√	0.589	0.150	32	0.437
+CS module +OC module	√	×	√	0.589	0.150	32	0.433
+CS module +OC module	√	√	×	0.593	0.220	47	0.406

pipeline	A	R	EAO	LN/次	V_FPS/(帧.s^-1)
SiamRPN++ +CS module	0.601	0.234	0.415	50	35
OC module +TU module	0.588	0.122	0.471	26	34

Tracker name	A	R	EAO	LN/次	V_FPS/(帧.s^-1)
SiamFC++	0.587	0.150	0.467	32	90
SiamBAN	0.590	0.178	0.447	38	40
DiMP50	0.597	0.153	0.440	33	43
SiamMargin	0.583	0.197	0.432	42	62
SiamRPN++	0.601	0.234	0.415	50	35
SiamR-CNN	0.612	0.22	0.405	47	15
ATOM	0.590	0.204	0.401	44	30
UpdateNet	0.587	0.276	0.393	59	92
DaSiamRPN	0.586	0.276	0.383	59	160
SiamMask	0.609	0.281	0.381	60	35
Ta-SiamRPN++	0.593	0.272	0.360	58	36
SiamDW	0.538	0.398	0.270	85	150
SiamFC	0.503	0.585	0.187	125	101
本文算法	0.588	0.122	0.471	26	34

在线目标分类及自适应模板更新的孪生网络跟踪算法

Tracking algorithm of Siamese network based on online target classification and adaptive template update

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 24

相关文章 15

Metrics

推荐阅读 0

[1]	曾浩, 母王强, 杨顺平. 高机动目标跟踪ATPM-IMM算法[J]. 通信学报, 2022, 43(7): 93-101.
[2]	张文林, 刘雪鹏, 牛铜, 陈琦, 屈丹. 基于正样本对比与掩蔽重建的自监督语音表示学习[J]. 通信学报, 2022, 43(7): 163-171.
[3]	熊金波, 周永洁, 毕仁万, 万良, 田有亮. 边缘协同的轻量级隐私保护分类框架[J]. 通信学报, 2022, 43(1): 127-137.
[4]	王洪雁, 张莉彬, 陈国强, 汪祖民, 管志远. 结合粒子滤波及度量学习的目标跟踪方法[J]. 通信学报, 2021, 42(5): 98-110.
[5]	程旭, 王莹莹, 张年杰, 付章杰, 陈北京, 赵国英. 基于空间感知的多级损失目标跟踪对抗攻击方法[J]. 通信学报, 2021, 42(11): 242-254.
[6]	柳超,张志国,孙进平. 基于幅度信息的海上多目标稳健跟踪方法[J]. 通信学报, 2019, 40(2): 60-69.
[7]	汪润,唐奔宵,王丽娜. DeepRD：基于Siamese LSTM网络的Android重打包应用检测方法[J]. 通信学报, 2018, 39(8): 69-82.
[8]	王亚文,陈鸿昶,李邵梅,高超. 融合遮挡感知的在线Boosting跟踪算法[J]. 通信学报, 2016, 37(9): 92-101.
[9]	卢旭,程良伦,罗世亮. 无线传感器网络自适应目标跟踪节点调度算法[J]. 通信学报, 2015, 36(4): 70-80.
[10]	罗会兰,杜芳芳,孔繁胜. 像素点特征加权的尺度自适应跟踪算法[J]. 通信学报, 2015, 36(10): 200-210.
[11]	蒋鹏1，宋华华1，林广2. 基于粒子群优化和M-H采样粒子滤波的传感器网络目标跟踪方法[J]. 通信学报, 2013, 34(11): 2-17.
[12]	蒋鹏,宋华华,林广. 基于粒子群优化和M-H抽样粒子滤波的传感器网络目标跟踪方法[J]. 通信学报, 2013, 34(11): 8-17.
[13]	王慧斌,陈哲,王鑫,马玉. 基于随机有限集的UPF-CPHD多目标跟踪[J]. 通信学报, 2012, 33(12): 147-153.
[14]	龙云利,盛卫东,徐晖,安玮. 锥扫型光学传感器像平面多目标轨迹跟踪[J]. 通信学报, 2011, 32(9): 123-128.
[15]	柳絮,李金宝,郭龙江,范斌,朱敬华. 传感器网络中一种能量有效的移动目标预测跟踪算法[J]. 通信学报, 2010, 31(9A): 135-143.

Tracker name	A	R	EAO	LN/次	V_FPS/(帧.s^-1)
SiamMargin	0.579	0.321	0.366	65	46
SiamBAN	0.602	0.396	0.327	79	40
DiMP	0.582	0.371	0.321	74	40
SiamDW_ST	0.600	0.467	0.299	93	—
SiamMask	0.594	0.461	0.287	92	35
SiamRPN++	0.599	0.482	0.285	96	35
ATOM	0.596	0.557	0.240	111	—
本文算法	0.579	0.296	0.337	59	33