基于原始点云网格自注意力机制的三维目标检测方法

doi:10.11959/j.issn.1000-436x.2023189

摘要/Abstract

摘要：

为了增强感兴趣区域（RoI）的特征表达，包括空间网格特征编码模块和软回归损失，提出了一种基于原始点云网格自注意力机制的三维目标检测方法GT3D。网格特征编码模块用于通过自注意力机制对点的局部特征和空间特征进行有效加权，充分考虑点云之间的几何关系，以提供更准确的特征表达；软回归损失用于改善数据标注过程中由于标注不准确而产生的回归歧义问题。将所提方法在公开的三维目标检测数据集KITTI上进行实验。结果表明，所提方法相比其他已公开的基于点云的三维目标检测方法检测准确率提升明显，并提交了KITTI官方测试集进行公开测试，对简单、中等和困难 3 个难度等级的汽车检测准确率分别达到 91.45%、82.76%和79.74%。

关键词: 三维目标检测, 点云, 自注意力机制, 空间坐标编码, 软回归损失

Abstract:

To enhance the feature representation of region of interest (RoI), which incorporated a spatial context encoding module and soft regression loss, a grid self-attention mechanism 3D object detection method based on raw point cloud, named GT3D, was proposed.The spatial context encoding module was designed to effectively weight the local and spatial features of points through the attention mechanism, considering the contribution of different point cloud features for a more accurate feature representation.The soft regression loss was introduced to address label ambiguity arising during the data annotation phase.Experiments conducted on the public KITTI 3D object detection dataset demonstrate that the proposed method achieves significant improvements in detection accuracy compared to other publicly available point cloud-based 3D object detection methods.The detection results of the test set are submitted to the official KITTI server for public evaluation, achieving detection accuracies of 91.45%, 82.76%, and 79.74% for easy, moderate, and hard difficulty levels in car detection, respectively.

Key words: 3D object detection, point cloud, self-attention mechanism, spatial coordinate encoding, soft regression loss

中图分类号:

TP391.4

鲁斌, 孙洋, 杨振宇. 基于原始点云网格自注意力机制的三维目标检测方法[J]. 通信学报, 2023, 44(10): 72-84.

Bin LU, Yang SUN, Zhenyu YANG. Grid self-attention mechanism 3D object detection method based on raw point cloud[J]. Journal on Communications, 2023, 44(10): 72-84.

图/表 18

图1

图2

图3

图4

图5

图6

图7

图8

表1

图9

表2

表3

表4

表5

图10

图11

表6

图12

参考文献 36

[1]	SHI S S , GUO C X , JIANG L ,et al. PV-RCNN:point-voxel feature set abstraction for 3D object detection[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 10526-10535.
[2]	YAN Y , MAO Y X , LI B . SECOND:sparsely embedded convolutional detection[J]. Sensors, 2018,18(10): 3337.
[3]	QI C R , YI L , SU H ,et al. PointNet++:deep hierarchical feature learning on point sets in a metric space[J]. arXiv Preprint,arXiv:1706.02413, 2017.
[4]	DENG J J , SHI S S , LI P W ,et al. Voxel R-CNN:towards high performance voxel-based 3D object detection[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park:AAAI Press, 2021: 1201-1209.
[5]	CHARLES R Q , HAO S , MO K C ,et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 77-85.
[6]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2017: 5998-6008.
[7]	GUO M H , CAI J X , LIU Z N ,et al. PCT:point cloud transformer[J]. Computational Visual Media, 2021,7(2): 187-199.
[8]	ZHAO H S , JIANG L , JIA J Y ,et al. Point transformer[C]// Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2022: 16239-16248.
[9]	ZHOU Y , TUZEL O . VoxelNet:end-to-end learning for point cloud based 3D object detection[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 4490-4499.
[10]	GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? the KITTI vision benchmark suite[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2012: 3354-3361.
[11]	LANG A H , VORA S , CAESAR H ,et al. PointPillars:fast encoders for object detection from point clouds[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 12689-12697.
[12]	SHI S S , WANG X G , LI H S . PointRCNN:3D object proposal generation and detection from point cloud[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 770-779.
[13]	YANG Z T , SUN Y N , LIU S ,et al. 3DSSD:point-based 3D single stage object detector[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 11037-11045.
[14]	QIAN R , LAI X , LI X R . BADet:boundary-aware 3D object detection from point clouds[J]. Pattern Recognition, 2022,125:108524.
[15]	ZHENG W , TANG W L , CHEN S J ,et al. CIA-SSD:confident IoU-aware single-stage object detector from point cloud[C]// Procee dings of the AAAI Conference on Artificial Intelligence. Menlo Park:AAAI Press, 2021: 3555-3562.
[16]	HU J S K , KUAI T S , WASLANDER S L . Point density-aware voxels for LiDAR 3D object detection[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2022: 8459-8468.
[17]	SHENGA H L , CAI S J , LIU Y ,et al. Improving 3D object detection with channel-wise transformer[C]// Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2022: 2723-2732.
[18]	CHEN Y L , LIU S , SHEN X Y ,et al. Fast point R-CNN[C]// Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2020: 9774-9783.
[19]	SHI S S , JIANG L , DENG J J ,et al. PV-RCNN++:point-voxel feature set abstraction with local vector representation for 3D object detection[J]. International Journal of Computer Vision, 2023,131(2): 531-551.
[20]	CARION N , MASSA F , SYNNAEVE G ,et al. End-to-end object detection with transformers[C]// European Conference on Computer Vision. Cham:Springer, 2020: 213-229.
[21]	ZHU X , SU W , LU L ,et al. Deformable DETR:deformable transformers for end-to-end object detection[C]// Proceedings of International Conference on Learning Representations.Singapore:OpenReview. net Press, 2021: 1-16.
[22]	CHEN X , KUNDU K , ZHU Y ,et al. 3D object proposals for accurate object class detection[C]// Proceedings of the 28th Conference on Neural Information Processing Systems. Massachusetts:MIT Press, 2015: 424-432.
[23]	LOSHCHILOV I , HUTTER F . SGDR:stochastic gradient descent with warm restarts[J]. arXiv Preprint,arXiv:1608.03983, 2016.
[24]	LIN T Y , GOYAL P , GIRSHICK R ,et al. Focal loss for dense object detection[C]// Proceedings of IEEE International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2017: 2999-3007.
[25]	QI C R , LIU W , WU C X ,et al. Frustum PointNets for 3D object detection from RGB-D data[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 918-927.
[26]	LIANG M , YANG B , WANG S L ,et al. Deep continuous fusion for multi-sensor 3D object detection[C]// European Conference on Computer Vision. Cham:Springer, 2018: 663-678.
[27]	ZHAO X , LIU Z , HU R ,et al. 3D object detection using scale invariant and feature reweighting networks[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park:AAAI Press, 2019: 9267-9274.
[28]	LIANG M , YANG B , CHEN Y ,et al. Multi-task multi-sensor fusion for 3D object detection[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 7337-7345.
[29]	YOO J H , KIM Y , KIM J ,et al. 3D-CVF:generating joint camera and LiDAR features using cross-view spatial feature fusion for 3D object Detection[C]// European Conference on Computer Vision. Cham:Springer, 2020: 720-736.
[30]	PANG S , MORRIS D , RADHA H . CLOCs:camera-LiDAR object candidates fusion for 3D object detection[C]// Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway:IEEE Press, 2020: 10386-10393.
[31]	YANG Z T , SUN Y N , LIU S ,et al. STD:sparse-to-dense 3D object detector for point cloud[C]// Proceedings of IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2019: 1951-1960.
[32]	LI J L , DAI H , SHAO L ,et al. From voxel to point:IoU-guided 3D object detection for point cloud with voxel-to-point decoder[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York:ACM Press, 2021: 4622-4631.
[33]	CHEN Y K , LI Y W , ZHANG X Y ,et al. Focal sparse convolutional networks for 3D object detection[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2022: 5418-5427.
[34]	YANG H H , LIU Z L , WU X P ,et al. Graph R-CNN:towards accurate 3D object detection with semantic-decorated local graph[C]// European Conference on Computer Vision. Cham:Springer, 2022: 662-679.
[35]	HE C H , ZENG H , HUANG J Q ,et al. Structure aware single-stage 3D object detection from point cloud[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 11870-11879.
[36]	CAI Q , PAN Y W , YAO T ,et al. 3D cascade RCNN:high quality object detection in point clouds[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2022,31: 5706-5719.

环境	参数
CPU	AMD EPYC 7543 32-Core Processor 15核
GPU	NVIDIA A40
显存	48 GB
操作系统	Ubuntu 20.04
Python版本	3.8
深度学习框架	PyTorch 1.8.1
CUDA版本	11.1
cuDNN版本	8.0
代码编辑环境	Visual Studio Code 1.66.1

模式	方法	简单	中等	困难	FPS
Image+Point Cloud	F-PointNet^[25]	82.19%	69.79%	60.59%	5.9
	ContFuse^[26]	83.68%	68.78%	61.67%	16.7
	PointSIFT+SENet^[27]	85.99%	72.72%	64.58%	—
	UberATG-MMF^[28]	88.40%	77.43%	70.22%	12.5
	3D-CVF at SPA^[29]	89.20%	80.05%	73.11%	13.3
	CLOCs^[30]	88.94%	80.67%	77.15%	—
Point-based	PointRCNN^[12]	86.96%	75.64%	70.70%	10
	STD^[31]	87.95%	79.71%	75.09%	12.5
	3DSSD^[13]	88.36%	79.57%	74.55%	26.3
	CT3D^[17]	87.83%	81.77%	77.16%	14.3
	PV-RCNN^[1]	90.25%	81.43%	76.82%	8.9
Voxel-based	VoxelNet^[1]	77.47%	65.11%	57.73%	4.4
	SECOND^[2]	83.34%	72.55%	65.82%	30.4
	PointPillars^[11]	82.58%	74.31%	68.99%	42.4
	Voxel R-CNN^[4]	90.90%	81.62%	77.06%	25.2
	FV2P^[32]	88.17%	81.81%	77.43%	8
	Focals Conv^[33]	90.20%	82.12%	77.50%	8.9
	GraR-Vo^[34]	91.29%	82.77%	77.20%	25.6
	PDV^[16]	90.43%	81.86%	77.36%	10
	SA-SSD^[35]	88.75%	79.79%	74.16%	25
	3D Cascade RCNN^[36]	90.46%	82.16%	77.31%	14.2
	GT3D	91.45%	82.76%	79.74%	15

模式	方法	简单	中等	困难
Point-based	PointRCNN^[12]	88.88%	78.63%	77.38%
	STD^[31]	89.70%	79.80%	79.30%
	3DSSD^[13]	89.71%	79.45%	78.67%
	CT3D^[17]	89.54%	86.06%	78.99%
	PV-RCNN^[1]	89.35%	83.69%	78.70%
Voxel-based	VoxelNet^[1]	81.97%	65.46%	62.85%
	SECOND^[2]	88.61%	78.62%	77.22%
	PointPillars^[11]	86.62%	76.06%	68.91%
	Voxel R-CNN^[4]	89.41%	84.52%	78.93%
	Focals Conv^[33]	89.52%	84.93%	79.18%
	PDV^[16]	89.52%	84.93%	79.18%
	SA-SSD^[35]	90.15%	79.91%	78.78%
	GT3D	89.78%	86.31%	79.22%

方法	简单	中等	困难
CT3D^[17]	91.99%	71.60%	67.34%
Voxel R-CNN^[4]	91.28%	72.54%	68.46%
PV-RCNN^[1]	88.88%	71.95%	66.78%
PDV^[16]	92.72%	74.23%	69.60%
GT3D	92.93%	74.65%	69.71%

阶段数	方法	mAP	参数量
单阶段	SECOND^[2]	73.90%	5.33×10⁶
	PointPillars^[11]	75.29%	4.83×10⁶
两阶段	PointRCNN^[12]	77.77%	4.04×10⁶
	3DSSD^[13]	80.83%	7.56×10⁶
	CT3D^[17]	82.25%	7.83×10⁶
	PV-RCNN^[1]	82.83%	13.12×10⁶
	Voxel R-CNN^[4]	83.19%	7.59×10⁶
	GT3D	84.65%	7.95×10⁶