基于差分特征注意力机制的无锚框多光谱行人检测算法

doi:10.11959/j.issn.2096-6652.202130

Abstract

Abstract:

Multispectral pedestrian detection system suffers with low feature fusion quality, high quantity of model hyper-parameters and complex anchor matching algorithm.To deal with these problems, an anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism was proposed.Firstly, differential modality aware fusion was used to obtain the complementary information between different modalities to optimize the channel features.Secondly, the CenterNet detection framework with anchor free mechanism was adopted to greatly reduce the computational complexity of the model and thus improve the detection speed.Finally, differential feature guided attention mechanism was introduced to improve the quality of feature fusion and further enhance the detection accuracy.Experimental results on three open datasets, KAIST, CVC14 and FLIR, show that the proposed algorithm can effectively improve the detection accuracy and speed compared with the current advanced methods, and has a good practical application prospect.

Key words: multispectral pedestrian detection, anchor free mechanism, CenterNet model, attention mechanism

CLC Number:

TP242

Jifeng SHEN, Yue1 LIU, Hao WEI, et al. Anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 294-303.

Figures/Tables 12

融合方案	全天	白天	夜间
仅用可见光	30.37%	29.62%	33.76%
仅用红外	28.21%	30.56%	26.91%
早期融合	23.58%	22.35%	23.12%
中期融合	19.41%	17.32%	21.28%
后期融合	$14 . 51 %$	$14 . 39 %$	$14 . 84 %$

上采样方法	MR^-2
基础算法	14.51%
双线性插值	13.43%
反卷积	$12 . 51 %$
上池化	14.22%

方法	MR^-2（全天）
基础算法	12.51%
基础算法+DMAF	10.50%
基础算法+DFA	11.12%
基础算法+DMAF+DFA	$9 . 48 %$

方法	全天	白天	夜间	近距离	中距离	远距离	无遮挡	部分遮挡	严重遮挡
ACF+C+T^[4]	47.32%	42.47%	56.17%	28.74%	53.67%	88.20%	62.94%	81.40%	88.08%
RPN+BF^[20]	18.29%	19.57%	16.27%	0.04%	30.87%	88.86%	47.45%	56.10%	72.20%
Halfway Fusion^[7]	25.75%	24.88%	26.59%	8.13%	30.34%	75.70%	43.13%	65.21%	74.36%
IAF^[21]	15.73%	14.55%	18.26%	0.96%	25.54%	77.84%	40.17%	48.40%	69.70%
IATDNN^[22]	14.95%	14.67%	15.72%	0.04%	28.55%	83.42%	45.43%	46.25%	64.57%
AR-CNN^[19]	10.43%	11.34%	8.85%	0.79%	$13 . 15 %$	$37 . 41 %$	$20 . 61 %$	$33 . 29 %$	61.42%
MBNet^[5]	$8 . 01 %$	$9 . 34 %$	$7 . 89 %$	0.12%	14.09%	53.75%	25.71%	35.24%	$58 . 16 %$
本文算法	8.76%	8.76%	8.71%	$0 . 01 %$	15.71%	51.14%	27.02%	33.93%	61.74%

References 23

[1]	郑南宁 . 人工智能新时代[J]. 智能科学与技术学报, 2019,1(1): 1-3.
	ZHENG N N . The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 1-3.
[2]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence is entering the post deep-learning era[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 4-6.
[3]	GRASSI A P , FROLOV V , LEóN F P , . Information fusion to detect and classify pedestrians using invariant features[J]. Information Fusion, 2011,12(4): 284-292.
[4]	HWANG S , PARK J , KIM N ,et al. Multispectral pedestrian detection:benchmark dataset and baseline[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 1037-1045.
[5]	ZHOU K L , CHEN L S , CAO X . Improving multispectral pedestrian detection by addressing modality imbalance problems[M]. Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 787-803.
[6]	J?RG W , VOLKER F , MICHAEL H ,et al. Multispectral pedestrian detection using deep fusion convolutional neural networks[C]// Proceedings of the 24th European Symposium on Artificial Neural Networks,Computational Intelligence and Machine Learning.[S.l.:s.n.], 2016.
[7]	LIU J J , ZHANG S T , WANG S ,et al. Multispectral deep neural networks for pedestrian detection[C]// Proceedings of the British Machine Vision Conference.[S.l.:s.n.], 2016: 731-733.
[8]	VANDERSTEEGEN M , BEECK K , GOEDEMé T , . Real-time multispectral pedestrian detection with a single-pass deep neural network[C]// Proceedings of the International Conference on Image Analysis and Recognition.[S.l.:s.n.], 2018: 419-426.
[9]	REDMON J , FARHADI A . YOLO9000:better,faster,stronger[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 6517-6525.
[10]	DENG J , DONG W , SOCHER R ,et al. ImageNet:a large-scale hierarchical image database[C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2009: 248-255.
[11]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014: 113-123.
[12]	LIN M , CHEN Q , YAN S C . Network in network[J]. Computer Science, 2013: 211-223.
[13]	SHEN J F , ZUO X , YANG W K ,et al. Differential features for pedestrian detection:a Taylor series perspective[J]. IEEE Transactions on Intelligent Transportation Systems, 2019,20(8): 2913-2922.
[14]	WOO S , PARK J , LEE J Y ,et al. CBAM:convolutional block attention module[M]. Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[15]	REN S Q , HE K M , GIRSHICK R ,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149.
[16]	ZHOU X Y , WANG D Q , PHILIPP K . Objects as points[J]. Communications and Computer Sciences, 2019: 232-239.
[17]	LIN T Y , GOYAL P , GIRSHICK R ,et al. Focal loss for dense object detection[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 2999-3007.
[18]	DAI J F , QI H Z , XIONG Y W ,et al. Deformable convolutional networks[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 764-773.
[19]	ZHANG L , ZHU X Y , CHEN X Y ,et al. Weakly aligned cross-modal learning for multispectral pedestrian detection[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 5126-5136.
[20]	K?NIG D , ADAM M , JARVERS C ,et al. Fully convolutional region proposal networks for multispectral person detection[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE Press, 2017: 243-250.
[21]	LI C Y , SONG D , TONG R F ,et al. Illumination-aware Faster R-CNN for robust multispectral pedestrian detection[J]. Pattern Recognition, 2019,85: 161-171.
[22]	GUAN D Y , CAO Y P , YANG J X ,et al. Fusion of multispectral data through illumination-aware deep neural networks for pedestrian detection[J]. Information Fusion, 2019,50: 148-157.
[23]	LIU S T , HUANG D , WANG Y H . Adaptive NMS:refining pedestrian detection in a crowd[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 6452-6461.

Metrics

Recommended 0

No Suggested Reading articles found!

Anchor free multispectral pedestrian detection algorithm based on differential feature attention mechanism

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 23

Related Articles 2

Metrics

Recommended 0

方法	检测速度/帧每秒
Faster R-CNN^[15]	10
MBNet^[5]	14
本文算法	22

[1]	Yan CHEN, Xueqin LUO, Wei LIANG, Yongfang XIE. Depression recognition based on emotional information fused with attentional mechanism [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(4): 600-609.
[2]	Kaishi YIN, Meng YANG, Xi GU, Zhicheng WANG. Light weight rotating object detector based on angle sensitive spatial attention mechanism [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(3): 322-333.

数据集	MR^-2
CVC-14	25.54%
FLIR	28.43%