基于差分特征注意力机制的无锚框多光谱行人检测算法

doi:10.11959/j.issn.2096-6652.202130

智能科学与技术学报 ›› 2021, Vol. 3 ›› Issue (3): 294-303.doi: 10.11959/j.issn.2096-6652.202130

• 专刊：目标智能检测与识别 • 上一篇下一篇

基于差分特征注意力机制的无锚框多光谱行人检测算法

沈继锋¹, 刘岳¹, 韦浩², 左欣³, 杨万扣⁴

¹ 江苏大学电气信息工程学院，江苏镇江 212013
² 中电海康集团有限公司，浙江杭州310000
³ 江苏科技大学计算机学院，江苏镇江 212003
⁴ 东南大学自动化学院，江苏南京 210003

修回日期:2021-08-12 出版日期:2021-09-15 发布日期:2021-09-01
作者简介:沈继锋（1980− ），男，博士，江苏大学电气信息工程学院副教授，主要研究方向为计算机视觉与模式识别
刘岳（1994− ），男，江苏大学电气信息工程学院硕士生，主要研究方向为计算机视觉
韦浩（1995− ），男，就职于中电海康集团有限公司，主要研究方向为计算机视觉
左欣（1980− ），女，博士，江苏科技大学计算机学院副教授，主要研究方向为计算机视觉与模式识别
杨万扣（1979− ），男，博士，东南大学自动化学院副研究员、博士生导师，主要研究方向为计算机视觉、模式识别与机器学习
基金资助:
国家自然科学基金资助项目(61903164);江苏省自然科学基金资助项目(BK20191427)

Anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism

Jifeng SHEN¹, Yue1 LIU¹, Hao WEI², Xin ZUO³, Wankou YANG⁴

¹ School of Electrical and Information Engineering, Jiangsu University, Zhenjiang 212013, China
² China Electric Haikang Group Co., Ltd., Hangzhou 310000, China
³ School of Computer Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang 212003, China
⁴ School of Automation, Southeast University, Nanjing 210003, China

Revised:2021-08-12 Online:2021-09-15 Published:2021-09-01
Supported by:
The National Natural Science Foundation of China(61903164);The Natural Science Foundation of Jiangsu Province(BK20191427)

摘要/Abstract

摘要：

针对多光谱行人检测系统存在特征融合质量低、模型超参数多且锚框匹配算法复杂等问题，提出了一种基于差分特征注意力机制的无锚框多光谱行人检测算法。该算法首先采用差分特征感知融合方法挖掘多模态特征间的互补信息来优化通道特征；然后利用具有高效无锚框机制的CenterNet检测框架大大降低了模型计算复杂度，从而提升检测速度；最后引入差分特征注意力机制，改善特征融合质量，进一步提升检测精度。在KAIST、CVC14和FLIR这3个公开数据集上的实验结果表明，提出的算法和其他先进方法相比，能够同时有效提升检测精度和速度，具有较好的实际应用前景。

关键词: 多光谱行人检测, 无锚框机制, CenterNet模型, 注意力机制

Abstract:

Multispectral pedestrian detection system suffers with low feature fusion quality, high quantity of model hyper-parameters and complex anchor matching algorithm.To deal with these problems, an anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism was proposed.Firstly, differential modality aware fusion was used to obtain the complementary information between different modalities to optimize the channel features.Secondly, the CenterNet detection framework with anchor free mechanism was adopted to greatly reduce the computational complexity of the model and thus improve the detection speed.Finally, differential feature guided attention mechanism was introduced to improve the quality of feature fusion and further enhance the detection accuracy.Experimental results on three open datasets, KAIST, CVC14 and FLIR, show that the proposed algorithm can effectively improve the detection accuracy and speed compared with the current advanced methods, and has a good practical application prospect.

Key words: multispectral pedestrian detection, anchor free mechanism, CenterNet model, attention mechanism

中图分类号:

TP242

沈继锋, 刘岳, 韦浩, 等. 基于差分特征注意力机制的无锚框多光谱行人检测算法[J]. 智能科学与技术学报, 2021, 3(3): 294-303.

Jifeng SHEN, Yue1 LIU, Hao WEI, et al. Anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 294-303.

图/表 12

图1

图2

图3

表1

特征融合消融实验结果"

融合方案	全天	白天	夜间
仅用可见光	30.37%	29.62%	33.76%
仅用红外	28.21%	30.56%	26.91%
早期融合	23.58%	22.35%	23.12%
中期融合	19.41%	17.32%	21.28%
后期融合	$14 . 51 %$	$14 . 39 %$	$14 . 84 %$

表1

表2

上采样方法消融实验结果"

上采样方法	MR^-2
基础算法	14.51%
双线性插值	13.43%
反卷积	$12 . 51 %$
上池化	14.22%

表2

表3

使用DMAF与DFA的消融实验结果"

方法	MR^-2（全天）
基础算法	12.51%
基础算法+DMAF	10.50%
基础算法+DFA	11.12%
基础算法+DMAF+DFA	$9 . 48 %$

表3

图4

表4

本文算法与其他先进算法在KAIST数据集上的实验结果"

方法	全天	白天	夜间	近距离	中距离	远距离	无遮挡	部分遮挡	严重遮挡
ACF+C+T^[4]	47.32%	42.47%	56.17%	28.74%	53.67%	88.20%	62.94%	81.40%	88.08%
RPN+BF^[20]	18.29%	19.57%	16.27%	0.04%	30.87%	88.86%	47.45%	56.10%	72.20%
Halfway Fusion^[7]	25.75%	24.88%	26.59%	8.13%	30.34%	75.70%	43.13%	65.21%	74.36%
IAF^[21]	15.73%	14.55%	18.26%	0.96%	25.54%	77.84%	40.17%	48.40%	69.70%
IATDNN^[22]	14.95%	14.67%	15.72%	0.04%	28.55%	83.42%	45.43%	46.25%	64.57%
AR-CNN^[19]	10.43%	11.34%	8.85%	0.79%	$13 . 15 %$	$37 . 41 %$	$20 . 61 %$	$33 . 29 %$	61.42%
MBNet^[5]	$8 . 01 %$	$9 . 34 %$	$7 . 89 %$	0.12%	14.09%	53.75%	25.71%	35.24%	$58 . 16 %$
本文算法	8.76%	8.76%	8.71%	$0 . 01 %$	15.71%	51.14%	27.02%	33.93%	61.74%

表4

图5

表5

表6

图6

参考文献 23

[1]	郑南宁 . 人工智能新时代[J]. 智能科学与技术学报, 2019,1(1): 1-3.
	ZHENG N N . The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 1-3.
[2]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence is entering the post deep-learning era[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 4-6.
[3]	GRASSI A P , FROLOV V , LEóN F P , . Information fusion to detect and classify pedestrians using invariant features[J]. Information Fusion, 2011,12(4): 284-292.
[4]	HWANG S , PARK J , KIM N ,et al. Multispectral pedestrian detection:benchmark dataset and baseline[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 1037-1045.
[5]	ZHOU K L , CHEN L S , CAO X . Improving multispectral pedestrian detection by addressing modality imbalance problems[M]. Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 787-803.
[6]	J?RG W , VOLKER F , MICHAEL H ,et al. Multispectral pedestrian detection using deep fusion convolutional neural networks[C]// Proceedings of the 24th European Symposium on Artificial Neural Networks,Computational Intelligence and Machine Learning.[S.l.:s.n.], 2016.
[7]	LIU J J , ZHANG S T , WANG S ,et al. Multispectral deep neural networks for pedestrian detection[C]// Proceedings of the British Machine Vision Conference.[S.l.:s.n.], 2016: 731-733.
[8]	VANDERSTEEGEN M , BEECK K , GOEDEMé T , . Real-time multispectral pedestrian detection with a single-pass deep neural network[C]// Proceedings of the International Conference on Image Analysis and Recognition.[S.l.:s.n.], 2018: 419-426.
[9]	REDMON J , FARHADI A . YOLO9000:better,faster,stronger[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 6517-6525.
[10]	DENG J , DONG W , SOCHER R ,et al. ImageNet:a large-scale hierarchical image database[C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2009: 248-255.
[11]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014: 113-123.
[12]	LIN M , CHEN Q , YAN S C . Network in network[J]. Computer Science, 2013: 211-223.
[13]	SHEN J F , ZUO X , YANG W K ,et al. Differential features for pedestrian detection:a Taylor series perspective[J]. IEEE Transactions on Intelligent Transportation Systems, 2019,20(8): 2913-2922.
[14]	WOO S , PARK J , LEE J Y ,et al. CBAM:convolutional block attention module[M]. Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[15]	REN S Q , HE K M , GIRSHICK R ,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149.
[16]	ZHOU X Y , WANG D Q , PHILIPP K . Objects as points[J]. Communications and Computer Sciences, 2019: 232-239.
[17]	LIN T Y , GOYAL P , GIRSHICK R ,et al. Focal loss for dense object detection[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 2999-3007.
[18]	DAI J F , QI H Z , XIONG Y W ,et al. Deformable convolutional networks[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 764-773.
[19]	ZHANG L , ZHU X Y , CHEN X Y ,et al. Weakly aligned cross-modal learning for multispectral pedestrian detection[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 5126-5136.
[20]	K?NIG D , ADAM M , JARVERS C ,et al. Fully convolutional region proposal networks for multispectral person detection[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE Press, 2017: 243-250.
[21]	LI C Y , SONG D , TONG R F ,et al. Illumination-aware Faster R-CNN for robust multispectral pedestrian detection[J]. Pattern Recognition, 2019,85: 161-171.
[22]	GUAN D Y , CAO Y P , YANG J X ,et al. Fusion of multispectral data through illumination-aware deep neural networks for pedestrian detection[J]. Information Fusion, 2019,50: 148-157.
[23]	LIU S T , HUANG D , WANG Y H . Adaptive NMS:refining pedestrian detection in a crowd[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 6452-6461.

基于差分特征注意力机制的无锚框多光谱行人检测算法

Anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 23

相关文章 2

Metrics

推荐阅读 0

方法	检测速度/帧每秒
Faster R-CNN^[15]	10
MBNet^[5]	14
本文算法	22

[1]	陈妍, 罗雪琴, 梁伟, 谢永芳. 基于情感信息融合注意力机制的抑郁症识别[J]. 智能科学与技术学报, 2022, 4(4): 600-609.
[2]	尹开石, 杨萌, 顾曦, 王志成. 基于角度敏感的空间注意力机制的轻量型旋转目标检测器[J]. 智能科学与技术学报, 2021, 3(3): 322-333.

数据集	MR^-2
CVC-14	25.54%
FLIR	28.43%