基于上下文注意力CNN的三维点云语义分割

doi:10.11959/j.issn.1000-436x.2020128

Abstract

Abstract:

Aiming at the under-segmentation of 3D point cloud semantic segmentation caused by the lack of contextual fine-grained information of the point cloud,an algorithm based on contextual attention CNN was proposed for 3D point cloud semantic segmentation.Firstly,the fine-grained features in local area of the point cloud were mined through the attention coding mechanism.Secondly,the contextual features between multi-scale local areas were captured by the contextual recurrent neural network coding mechanism and compensated with the fine-grained local features.Finally,the multi-head mechanism was used to enhance the generalization ability of the network.Experiments show that the mIoU of the proposed algorithm on the three standard datasets of ShapeNet Parts,S3DIS and vKITTI are 85.4%,56.7% and 38.1% respectively,which has good segmentation performance and good generalization ability.

Key words: 3D point cloud, semantic segmentation, contextual attention convolution layer, convolutional neural network, deep learning

CLC Number:

TP391

Jun YANG,Jisheng DANG. Semantic segmentation of 3D point cloud based on contextual attention CNN[J]. Journal on Communications, 2020, 41(7): 195-203.

Figures/Tables 12

References 28

[1]	KRIZHEVSKY A , SUTSKEVER I , HINTON G . ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems. Piscataway:IEEE Press, 2012: 1097-1105.
[2]	YI L , SU H , GUO X ,et al. SyncSpecCNN:synchronized spectral CNN for 3D shape segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 2282-2290.
[3]	ZHANG J , ZHENG J , WU C ,et al. Variational mesh decomposition[J]. ACM Transactions on Graphics (TOG), 2012,31(21): 1-14.
[4]	XIAO D , LIN H , XIAN C ,et al. CAD mesh model segmentation by clustering[J]. Computers ＆ Graphics, 2011,35(3): 685-691.
[5]	KU J , MOZIFIAN M , LEE J ,et al. Joint 3D proposal generation and object detection from view aggregation[C]// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2018: 1-8.
[6]	YANG B , LUO W , URTASUN R . PIXOR:real-time 3D object detection from point clouds[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 7652-7660.
[7]	KALOGERAKIS E , AVERKIOU M , MAJI S ,et al. 3D shape segmentation with projective convolutional networks[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 3779-3788.
[8]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. arXiv Preprint,arXiv:1409.1556, 2014
[9]	HUANG H , KALOGERAKIS E , CHAUDHURI S ,et al. Learning local shape descriptors from part correspondences with multi view convolutional networks[J]. ACM Transactions on Graphics (TOG), 2018,37(1): 1-14.
[10]	CHEN X , MA H , WAN J ,et al. Multi-view 3D object detection network for autonomous driving[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 1907-1915.
[11]	QI C , LIU W , WU C ,et al. Frustum PointNets for 3D object detection from RGB-D data[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 918-927.
[12]	WU Z , SONG S , KHOSLA A ,et al. 3D ShapeNets:a deep representation for volumetric shapes[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 1912-1920.
[13]	KLOKOV R , LEMPITSKY V . Escape from cells:deep Kd-networks for the recognition of 3D point cloud models[C]// IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 863-872.
[14]	TATARCHENKO M , DOSOVITSKIY A , BROX T . Octree generating networks:efficient convolutional architectures for high-resolution 3D outputs[C]// IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 2088-2096.
[15]	QI C , SU H , MO K ,et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 652-660.
[16]	QI C , YI L , SU H ,et al. PointNet++:deep hierarchical feature learning on point sets in a metric space[C]// Advances in Neural Information Processing Systems. Piscataway:IEEE Press, 2017: 5099-5108.
[17]	WANG Y , SUN Y , LIU Z ,et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019,38(5): 146-160.
[18]	CHEN C , FRAGONARA L , TSOURDOS A . GAPNet:graph attention based point neural network for exploiting local feature of point cloud[J]. arXiv Preprint,arXiv:1905.08705, 2019
[19]	LIU X , HAN Z , LIU Y ,et al. Point2sequence:learning the shape representation of 3D point clouds with an attention-based sequence to sequence network[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2019,33: 8778-8785.
[20]	WANG W , YU R , HUANG Q ,et al. SGPN:similarity group proposal network for 3D point cloud instance segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 2569-2578.
[21]	JIANG M , WU Y , ZHAO T ,et al. PointSIFT:a SIFT-like network module for 3D point cloud semantic segmentation[J]. arXiv Preprint,arXiv:1807.00652, 2018
[22]	YE X , LI J , HUANGU H ,et al. 3D recurrent neural networks with context fusion for point cloud semantic segmentation[C]// Proceedings of the European Conference on Computer Vision (ECCV). Piscataway:IEEE Press, 2018: 403-417.
[23]	LANDRIEU L , SIMONOVSKY M . Large-scale point cloud semantic segmentation with superpoint graphs[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 4558-4567.
[24]	LIN M , CHEN Q , YAN S . Network in network[J]. arXiv Preprint,arXiv:1312.4400, 2013
[25]	YI L , GUIBAS L , KIM V ,et al. A scalable active framework for region annotation in 3D shape collections[J]. ACM Transactions on Graphics, 2016,35(6): 1-12.
[26]	ARMENI I , SENER O , ZAMIR A ,et al. 3D semantic parsing of large-scale indoor spaces[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 1534-1543.
[27]	GAIDON A , WANG Q , CABON Y ,et al. Virtual worlds as proxy for multi-object tracking analysis[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 4340-4349.
[28]	ENGELMANN F , KONTOGIANNI T , HERMANS A ,et al. Exploring spatial context for 3D semantic segmentation of point clouds[C]// IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 716-724.

Metrics

Recommended 0

No Suggested Reading articles found!

层名称	输入通道	输出通道	卷积核大小	卷积步长	批规范化	激活函数
Layer₁	179	64	1×1	1	是	Selu
Layer₂	243	64	1×1	1	是	Selu
Layer₃	128	64	1×1	1	是	Selu
Layer₄	128	64	1×1	1	是	Selu
Layer₅	128	128	1×1	1	是	Selu
Layer₆	352	1 024	1×1	1	否	Selu

算法	mIoU	前向传播时间/ms
Kd-Net^[13]	82.3%	—
PointNet^[15]	83.7%	16.9
PointNet++^[16]	85.1%	30.7
DGCNN^[17]	85.1%	40.4
本文算法	85.4%	39.0

算法	mIoU
PointNet^[15]（自编码）	83.7%
本文算法（注意力编码）	84.4%
本文算法（注意力编码+上下文RNN编码）	85.4%

组件	mIoU
PointNet无组件^[16]	83.7%
CAC	85.1%
M-CAC (M=3，dropout=0.7)M-CAC (M=4，dropout=0.6)	85.3% 84.9%
M-CAC+残差学习	85.4%

算法	mIoU	OA
PointNet^[15]	47.6%	78.5%
MS+CU^[28]	47.8%	79.2%
G+RCU^[28]	49.7%	81.1%
SGPN^[20]	50.4%	80.8%
PointNet++ ^[16]	54.5%	81.0%
DGCNN ^[17]	56.1%	84.1%
本文算法	57.6%	85.2%

Semantic segmentation of 3D point cloud based on contextual attention CNN

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 28

Related Articles 15

Metrics

Recommended 0

[1]	Dongyu CHEN, Hua CHEN, Limin FAN, Yifang FU, Jian WANG. Research on test strategy for randomness based on deep learning [J]. Journal on Communications, 2023, 44(6): 23-33.
[2]	Rongpeng LI, Bingyan WANG, Honggang ZHANG, Zhifeng ZHAO. Design of knowledge enhanced semantic communication receiver [J]. Journal on Communications, 2023, 44(6): 70-76.
[3]	Shuai MA, Ke PEI, Huayan QI, Hang LI, Wen CAO, Hongmei WANG, Hailiang XIONG, Shiyin LI. Research on geomagnetic indoor high-precision positioning algorithm based on generative model [J]. Journal on Communications, 2023, 44(6): 211-222.
[4]	Jie YANG, Biao DONG, Xue FU, Yu WANG, Guan GUI. Lightweight decentralized learning-based automatic modulation classification method [J]. Journal on Communications, 2022, 43(7): 134-142.
[5]	Xiuzhang YANG, Guojun PENG, Zichuan LI, Yangqi LYU, Side LIU, Chenguang LI. Research on entity recognition and alignment of APT attack based on Bert and BiLSTM-CRF [J]. Journal on Communications, 2022, 43(6): 58-70.
[6]	Yong LIAO, Shiyi WANG. CSI feedback algorithm based on RM-Net for massive MIMO systems in high-speed mobile environment [J]. Journal on Communications, 2022, 43(5): 166-176.
[7]	Yurong LIAO, Haining WANG, Cunbao LIN, Yang LI, Yuqiang FANG, Shuyan NI. Research progress of deep learning-based object detection of optical remote sensing image [J]. Journal on Communications, 2022, 43(5): 190-203.
[8]	Fan ZHANG, Yun HUANG, Zizhuo FANG, Wei GUO. Lost-minimum post-training parameter quantization method for convolutional neural network [J]. Journal on Communications, 2022, 43(4): 114-122.
[9]	Zenghua ZHAO, Yuefan TONG, Jiayang CUI. Device-independent Wi-Fi fingerprinting indoor localization model based on domain adaptation [J]. Journal on Communications, 2022, 43(4): 143-153.
[10]	Zhengyu ZHU, Gengwang HOU, Chongwen HUANG, Gangcan SUN, Wanming HAO, Jing LIANG. Systems resource allocation algorithm for RIS-assisted D2D secure communication based on parallel CNN [J]. Journal on Communications, 2022, 43(3): 172-179.
[11]	Yong LIAO, Gang CHENG, Yujie LI. CSI feedback algorithm based on deep unfolding for massive MIMO systems [J]. Journal on Communications, 2022, 43(12): 77-88.
[12]	Zeyu WANG, Shuhui BU, Wei HUANG, Yuanpan ZHENG, Qinggang WU, Huawen CHANG, Xu ZHANG. Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features [J]. Journal on Communications, 2022, 43(12): 157-171.
[13]	Xueyuan DUAN, Yu FU, Kun WANG, Bin LI. LDoS attack detection method based on simple statistical features [J]. Journal on Communications, 2022, 43(11): 53-64.
[14]	Junyan HUO, Ruipeng QIU, Yanzhuo MA, Fuzheng YANG. Reference frame list optimization algorithm in video coding by quality enhancement of the nearest picture [J]. Journal on Communications, 2022, 43(11): 136-147.
[15]	Haiyan KANG, Yuanrui JI. Research on federated learning approach based on local differential privacy [J]. Journal on Communications, 2022, 43(10): 94-105.