基于卷积神经网络的交通场景语义分割方法研究

doi:10.11959/j.issn.1000-436x.2018053

通信学报 ›› 2018, Vol. 39 ›› Issue (4): 123-130.doi: 10.11959/j.issn.1000-436x.2018053

基于卷积神经网络的交通场景语义分割方法研究

李琳辉,钱波,连静,郑伟娜,周雅夫

大连理工大学工业装备结构分析国家重点实验室运载工程与力学学部汽车工程学院，辽宁大连 116024

出版日期:2018-04-01 发布日期:2018-04-29
作者简介:李琳辉（1981-），男，河南辉县人，博士，大连理工大学副教授，主要研究方向为汽车安全辅助驾驶、智能车辆及基于视觉传感器的车载环境感知等。|钱波（1991-），男，江苏宿迁人，大连理工大学硕士生，主要研究方向为图像语义分割、立体视觉。|连静（1980-），女，吉林公主岭人，博士，大连理工大学副教授，主要研究方向为智能电动汽车、新能源汽车动力总成及整车控制等。|郑伟娜（1994-），女，山东日照人，大连理工大学硕士生，主要研究方向为交通场景语义分割、图像理解。|周雅夫（1962-），男，辽宁铁岭人，大连理工大学教授，主要研究方向为新能源汽车智能化技术、车载信息采集与远程监控技术、电动汽车整车匹配设计与控制技术、电动汽车驱动电机及其控制技术等。
基金资助:
国家自然科学基金资助项目(51775082);国家自然科学基金资助项目(61473057);国家自然科学基金资助项目(61203171);中央高校基本科研业务费专项基金资助项目(DUT15LK13);中央高校基本科研业务费专项基金资助项目(DUT17LAB11)

Study on traffic scene semantic segmentation method based on convolutional neural network

Linhui LI,Bo QIAN,Jing LIAN,Weina ZHENG,Yafu ZHOU

School of Automotive Engineering,Faculty of Vehicle Engineering and Mechanics,State Key Laboratory of Structural Analysis for Industrial Equipment,Dalian University of Technology,Dalian 116024,China

Online:2018-04-01 Published:2018-04-29
Supported by:
The National Natural Science Foundation of China(51775082);The National Natural Science Foundation of China(61473057);The National Natural Science Foundation of China(61203171)

摘要/Abstract

摘要：

为提高交通场景的语义分割精度，提出一种基于 RGB-D 图像和卷积神经网络的分割方法。首先，基于半全局立体匹配算法获取视差图D，并将其与RGB图像融合成四通道RGB-D图像，以建立样本库；其次，对于2种不同结构的卷积神经网络，分别采用2种不同的学习率调整策略对网络进行训练；最后，对训练得到的网络进行测试及对比分析。实验结果表明，基于RGB-D图像的交通场景语义分割算法得到的分割精度高于基于RGB图像的分割算法。

关键词: 深度学习, 卷积神经网络, 交通场景, 语义分割, 视差图

Abstract:

In order to improve the semantic segmentation accuracy of traffic scene,a segmentation method was proposed based on RGB-D image and convolutional neural network.Firstly,on the basis of semi-global stereo matching algorithm,the disparity map was obtained,and the sample library was established by fusing the disparity map D and RGB image into the four-channel RGB-D image.Then,with two different structures,the networks were trained by using two different learning rate adjustment strategy respectively.Finally,the traffic scene semantic segmentation test was carried out with RGB-D image as the input,and the results were compared with the segmentation method based on RGB image.The experimental results show that the proposed traffic scene segmentation algorithm based on RGB-D image can achieve higher semantic segmentation accuracy than that based on RGB image.

Key words: deep learning, convolutional neural network, traffic scene, semantic segmentation, disparity map

中图分类号:

U495，TP391.4

李琳辉,钱波,连静,郑伟娜,周雅夫. 基于卷积神经网络的交通场景语义分割方法研究[J]. 通信学报, 2018, 39(4): 123-130.

Linhui LI,Bo QIAN,Jing LIAN,Weina ZHENG,Yafu ZHOU. Study on traffic scene semantic segmentation method based on convolutional neural network[J]. Journal on Communications, 2018, 39(4): 123-130.

图/表 6

图1

图2

表1

表2

图3

图4

参考文献 25

[1]	ANBALAGAN T , GOWRISHANKAR C , SHANMUGAM A . SVM based road surface detection to improve performance of ABS[J]. Journal of Theoretical ＆ Applied Information Technology, 2013,51(2): 234-239.
[2]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553): 436-444.
[3]	高常鑫, 桑农 . 基于深度学习的高分辨率遥感影像目标检测[J]. 测绘通报, 2014(S1): 108-111.
	GAO C X , SANG N . Deep learning for object detection in remote sensing image[J]. Bulletin of Surveying and Mapping, 2014(S1): 108-111.
[4]	高凯珺, 孙韶媛, 姚广顺 ,等. 基于深度学习的无人车夜视图像语义分割[J]. 应用光学, 2017,38(3): 421-428.
	GAO K J , SUN S Y , YAO G S ,et al. Semantic segmentation of night vision images for unmanned vehicles based on deep learning[J]. Journal of Applied Optics, 2017,38(3): 421-428.
[5]	刘丹, 刘学军, 王美珍 . 一种多尺度 CNN 的图像语义分割算法[J]. 遥感信息, 2017,32(1): 57-64.
	LIU D , LIU X J , WANG M Z . Semantic segmentation with multi-scale convolutional neural network[J]. Remote Sensing Information, 2017,32(1): 57-64.
[6]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012,25(2): 1-9.
[7]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. ArXiv Preprint,ArXiv:1409.1556, 2014.
[8]	SZEGEDY C , LIU W , JIA Y ,et al. Going deeper with convolutions[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2014: 1-9.
[9]	DENG J , DONG W , SOCHER R ,et al. ImageNet:a large-scale hierarchical image database[C]// IEEE Computer Vision and Pattern Recognition. 2009: 248-255.
[10]	LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[C]// IEEE Computer Vision and Pattern Recognition. 2015: 3431-3440.
[11]	BADRINARAYANAN V , HANDA A , CIPOLLA R . SegNet:a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[J]. ArXiv Preprint,ArXiv:1505.07293, 2015.
[12]	BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis ＆ Machine Intelligence, 2017,PP(99): 1.
[13]	NOH H , HONG S , HAN B . Learning deconvolution network for semantic segmentation[C]// IEEE International Conference on Computer Vision. 2015: 1520-1528.
[14]	SILBERMAN N , HOIEM D , KOHLI P ,et al. Indoor segmentation and support inference from RGBD images[C]// European Conference on Computer Vision. 2012: 746-760.
[15]	GUPTA S , GIRSHICK R , ARBELáEZ P ,et al. Learning rich features from RGB-D images for object detection and segmentation[C]// European Conference on Computer Vision. 2014: 345-360.
[16]	SHAO T , XU W , ZHOU K ,et al. An interactive approach to semantic modeling of indoor scenes with an RGBD camera[J]. ACM Transactions on Graphics, 2012,31(6): 439-445.
[17]	FILLIAT D , BATTESTI E , BAZEILLE S ,et al. RGBD object recognition and visual texture classification for indoor semantic mapping[C]// 2012 IEEE International Conference on Technologies for Practical Robot Applications. 2012: 127-132.
[18]	GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2012: 3354-3361.
[19]	LI L , HUANG H , QIAN B ,et al. Vehicle detection method based on mean shift clustering[J]. Journal of Intelligent ＆ Fuzzy Systems, 2016,31(3): 1355-1363.
[20]	MIN D , CHOI S , LU J ,et al. Fast global image smoothing based on weighted least squares[J]. IEEE Transactions on Image Processing a Publication of the IEEE Signal Processing Society, 2014,23(12): 5638-5653.
[21]	RUSSELL B C , TORRALBA A , MURPHY K P ,et al. LabelMe:a database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008,77(1-3): 157-173.
[22]	GOULD S , FULTON R , KOLLER D . Decomposing a scene into geometric and semantically consistent regions[C]// IEEE International Conference on Computer Vision. 2009: 1-8.
[23]	EIGEN D , FERGUS R . Predicting depth,surface normals and semantic labels with a common multi-scale convolutional architecture[C]// IEEE International Conference on Computer Vision. 2015: 2650-2658.
[24]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[25]	JIA Y , SHELHAMER E , DONAHUE J ,et al. Caffe:convolutional architecture for fast feature embedding[C]// The 22nd ACM International Conference on Multimedia. 2014: 675-678.

网络模型	图像格式				分割精度				平均精确率	全局精确率
网络模型	图像格式	天空	建筑	路面	人行道	树木	草坪	车辆	平均精确率	全局精确率
SegNet	RGB	0.952	0.744	0.964	0.658	0.877	0.562	0.880	0.805	0.858
	RGB-D	0.964	0.778	0.969	0.661	0.869	0.597	0.911	0.821	0.875
SegNet-Basic	RGB	0.941	0.815	0.946	0.677	0.815	0.630	0.856	0.811	0.859
	RGB-D	0.937	0.866	0.956	0.768	0.792	0.709	0.892	0.846	0.870

网络模型	图像格式				分割精度				平均精确率	全局精确率
网络模型	图像格式	天空	建筑	路面	人行道	树木	草坪	车辆	平均精确率	全局精确率
SegNet	RGB	0.946	0.779	0.941	0.652	0.830	0.533	0.844	0.789	0.842
	RGB-D	0.949	0.809	0.944	0.677	0.812	0.522	0.877	0.799	0.852
SegNet-Basic	RGB	0.943	0.783	0.902	0.700	0.818	0.735	0.865	0.821	0.840
	RGB-D	0.936	0.789	0.915	0.757	0.830	0.718	0.866	0.830	0.855

基于卷积神经网络的交通场景语义分割方法研究

Study on traffic scene semantic segmentation method based on convolutional neural network

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 25

相关文章 15

Metrics

推荐阅读 0

[1]	陈东昱, 陈华, 范丽敏, 付一方, 王舰. 基于深度学习的随机性检验策略研究[J]. 通信学报, 2023, 44(6): 23-33.
[2]	李荣鹏, 汪丙炎, 张宏纲, 赵志峰. 知识增强的语义通信接收端设计[J]. 通信学报, 2023, 44(6): 70-76.
[3]	马帅, 裴科, 祁华艳, 李航, 曹雯, 王洪梅, 熊海良, 李世银. 基于生成模型的地磁室内高精度定位算法研究[J]. 通信学报, 2023, 44(6): 211-222.
[4]	杨洁, 董标, 付雪, 王禹, 桂冠. 基于轻量化分布式学习的自动调制分类方法[J]. 通信学报, 2022, 43(7): 134-142.
[5]	李昂, 陈建新, 魏昕, 周亮. 面向6G的跨模态信号重建技术[J]. 通信学报, 2022, 43(6): 28-40.
[6]	杨秀璋, 彭国军, 李子川, 吕杨琦, 刘思德, 李晨光. 基于Bert和BiLSTM-CRF的APT攻击实体识别及对齐研究[J]. 通信学报, 2022, 43(6): 58-70.
[7]	王晓丹, 李京泰, 宋亚飞. DDAC：面向卷积神经网络图像隐写分析模型的特征提取方法[J]. 通信学报, 2022, 43(5): 68-81.
[8]	廖勇, 王世义. 高速移动环境下基于RM-Net的大规模MIMO CSI反馈算法[J]. 通信学报, 2022, 43(5): 166-176.
[9]	廖育荣, 王海宁, 林存宝, 李阳, 方宇强, 倪淑燕. 基于深度学习的光学遥感图像目标检测研究进展[J]. 通信学报, 2022, 43(5): 190-203.
[10]	张帆, 黄赟, 方子茁, 郭威. 卷积神经网络的损失最小训练后参数量化方法[J]. 通信学报, 2022, 43(4): 114-122.
[11]	赵增华, 童跃凡, 崔佳洋. 基于域自适应的Wi-Fi指纹设备无关室内定位模型[J]. 通信学报, 2022, 43(4): 143-153.
[12]	朱政宇, 侯庚旺, 黄崇文, 孙钢灿, 郝万明, 梁静. 基于并行CNN的RIS辅助D2D保密通信系统资源分配算法[J]. 通信学报, 2022, 43(3): 172-179.
[13]	王泽宇, 布树辉, 黄伟, 郑远攀, 吴庆岗, 常化文, 张旭. 基于多模态特征的无监督领域自适应多级对抗语义分割网络[J]. 通信学报, 2022, 43(12): 157-171.
[14]	廖勇, 程港, 李玉杰. 基于深度展开的大规模MIMO系统CSI反馈算法[J]. 通信学报, 2022, 43(12): 77-88.
[15]	段雪源, 付钰, 王坤, 李彬. 基于简单统计特征的LDoS攻击检测方法[J]. 通信学报, 2022, 43(11): 53-64.