基于CNN的双边融合网络在高光谱图像分类中的应用

doi:10.11959/j.issn.1000-436x.2020238

通信学报 ›› 2020, Vol. 41 ›› Issue (11): 132-140.doi: 10.11959/j.issn.1000-436x.2020238

基于CNN的双边融合网络在高光谱图像分类中的应用

高红民,曹雪莹,杨耀,花再军,李臣明()

河海大学计算机与信息学院，江苏南京 211100

修回日期:2020-09-07 出版日期:2020-11-25 发布日期:2020-12-19
作者简介:高红民（1983- ），男，江苏仪征人，博士，河海大学教授，主要研究方向为深度学习等智能计算方法、图像大数据与人工智能、遥感影像处理等|曹雪莹（1994- ），女，江苏睢宁人，河海大学硕士生，主要研究方向为深度学习与图像处理等|杨耀（1995- ），男，江西宜春人，河海大学硕士生，主要研究方向为深度学习与图像处理等|花再军（1983- ），男，江苏姜堰人，河海大学实验师，主要研究方向为机器学习及图像处理等|李臣明（1969- ），男，内蒙古通辽人，河海大学教授，主要研究方向为智能信息处理、遥感技术与系统等
基金资助:
国家自然科学基金资助项目(61701166);中央高校基本科研业务费基金资助项目(B200202183);江苏省研究生科研与实践创新计划基金资助项目(SJCX20_0181);国家重点研发计划基金资助项目(2018YFC1508106)

Application of bilateral fusion model based on CNN in hyperspectral image classification

Hongmin GAO,Xueying CAO,Yao YANG,Zaijun HUA,Chenming LI()

College of Computer and Information Engineering,Hohai University,Nanjing 211100,China

Revised:2020-09-07 Online:2020-11-25 Published:2020-12-19
Supported by:
The National Natural Science Foundation of China(61701166);The Fundamental Research Funds for the Central Universities of China(B200202183);Postgraduate Research ＆ Practice Innovation Program of Jiangsu Province(SJCX20_0181);The National Key Research and Development Program of China(2018YFC1508106)

摘要/Abstract

摘要：

针对基于深度卷积神经网络的高光谱图像分类算法中存在的空间分辨率下降、池化操作引发特征丢失从而导致分类精度下降的问题，设计了一种由双边融合块构成的双边融合块网络。1×1卷积与超链接构成双边融合块上结构，传递局部空间特征，池化、卷积、反卷积、上采样组成下结构，强化高效判别特征。在3个基准高光谱图像数据集上的实验结果表明，该模型优于其他同类分类模型。

关键词: 卷积神经网络, 高光谱图像分类, 转置卷积, 上采样, 超链接

Abstract:

Aiming at the issues of decreasing spatial resolution and feature loss caused by pooling operation in depth CNN-based hyperspectral image classification algorithm,a bilateral fusion block network （DFBN）composed of bilateral fusion blocks was designed.The upper structure of bilateral fusion block was constituted by 1×1 convolution and hyperlink,which was used to transfer local spatial characteristics.The lower structure was constituted by pooling layer,convolutional layer,deconvolution layer and upsampling to enhance the characteristics of efficient discrimination.Experimental results on three benchmark hyperspectral image data sets illustrate that the model is superior to other similar classification models.

Key words: convolutional neural network, hyperspectral images classification, transpose-convolution, upsampling,hyperlink

中图分类号:

TP302

高红民,曹雪莹,杨耀,花再军,李臣明. 基于CNN的双边融合网络在高光谱图像分类中的应用[J]. 通信学报, 2020, 41(11): 132-140.

Hongmin GAO,Xueying CAO,Yao YANG,Zaijun HUA,Chenming LI. Application of bilateral fusion model based on CNN in hyperspectral image classification[J]. Journal on Communications, 2020, 41(11): 132-140.

图/表 18

图1

图3

图2

图4

表1

表2

表3

表4

表5

图5

表6

表7

表9

图6

图7

表8

表10

表11

参考文献 32

[1]	LUO B , YANG C H , CHANUSSOT J ,et al. Crop yield estimation based on unsupervised linear unmixing of multidate hyperspectral imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013,51(1): 162-173.
[2]	YUEN P W T , RICHARDSON M . An introduction to hyperspectral imaging and its application for security,surveillance and target acquisition[J]. Imaging Science Journal, 2010,58(5): 241-253.
[3]	SANDIDGE J C , HOLYER R J . Coastal bathymetry from hyperspectral observations of water radiance[J]. Remote Sensing of Environment, 1998,65(3): 341-352.
[4]	ZHANG L F , ZHANG L P , TAO D C ,et al. Hyperspectral remote sensing image subpixel target detection based on supervised metric learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014,52(8): 4955-4965.
[5]	LI J , BIOUCAS-DIAS J M , PLAZA A . Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010,48(11): 4085-4098.
[6]	SAMANIEGO L , BARDOSSY A , SCHULZ K . Supervised classification of remotely sensed imagery using a modified KNN technique[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008,46(7): 2112-2125.
[7]	MELGANI F , BRUZZONE L . Classification of hyperspectral remote sensing images with support vector machines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2004,42(8): 1778-1790.
[8]	WANG Q , LIN J Z , YUAN Y . Salient band selection for hyperspectral image classification via manifold ranking[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016,27(6): 1279-1289.
[9]	JIA S , TANG G H , ZHU J S ,et al. A novel ranking-based clustering approach for hyperspectral band selection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016,54(1): 88-102.
[10]	LICCIARDI G , MARPU P R , CHANUSSOT J ,et al. Linear versus nonlinear PCA for the classification of hyperspectral data based on the extended morphological profiles[J]. IEEE Geoscience and Remote Sensing Letters, 2012,9(3): 447-451.
[11]	VILLA A , BENEDIKTSSON J A , CHANUSSOT J ,et al. Hyperspectral image classification with independent component discriminant analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011,49(12): 4865-4876.
[12]	LIU X , ZHANG B , GAO L R ,et al. A maximum noise fraction transform with improved noise estimation for hyperspectral images[J]. Science in China Series F-information Sciences, 2009,52(9): 1578-1587.
[13]	MAKANTASIS K , KARANTZALOS K , DOULAMIS A ,et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]// 2015 IEEE International Geoscience and Remote Sensing Symposium. Piscataway:IEEE Press, 2015: 4959-4962.
[14]	GHAMISI P , PLAZA J , CHEN Y S ,et al. Advanced spectral classifiers for hyperspectral images:a review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017,5(1): 8-32.
[15]	GAO W , PENG Y . Ideal kernel-based multiple kernel learning for spectral-spatial classification of hyperspectral image[J]. IEEE Geoscience and Remote Sensing Letters, 2017,14(7): 1051-1055.
[16]	CHEN Y , NASRABADI N M , TRAN T D . Hyperspectral image classification via kernel sparse representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013,51(1): 217-231.
[17]	CHEN P , NELSON J D B , TOURNERET J Y . Toward a sparse Bayesian Markov random field approach to hyperspectral unmixing and classification[J]. IEEE Transactions on Image Processing, 2017,26(1): 426-438.
[18]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553): 436-444.
[19]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[20]	GIRSHICK R , DONAHUE J , DARRELL T ,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 580-587.
[21]	CHEN Y S , LIN Z H , ZHAO X ,et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014,7(6): 2094-2107.
[22]	LIU P , ZHANG H , EOM K B . Active deep learning for classification of hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017,10(2): 712-724.
[23]	GHAMISI P , MAGGIORI E , LI S T ,et al. New frontiers in spectral-spatial hyperspectral image classification the latest advances based on mathematical morphology,Markov random fields,segmentation,sparse representation,and deep learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(3): 10-43.
[24]	LI S T , SONG W W , FANG L Y ,et al. Deep learning for hyperspectral image classification:an overview[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(9): 6690-6709.
[25]	ZHONG Z L , LI J , LUO Z M ,et al. Spectral-spatial residual network for hyperspectral image classification:a 3D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(2): 847-858.
[26]	FENG F , WANG S T , WANG C Y ,et al. Learning deep hierarchical spatial-spectral features for hyperspectral image classification based on residual 3D-2D CNN[J]. Sensors, 2019,19(23):5276.
[27]	HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 770-778.
[28]	LI J J , ZHAO X , LI Y S ,et al. Classification of hyperspectral imagery using a new fully convolutional neural network[J]. IEEE Geoscience and Remote Sensing Letters, 2018,15(2): 292-296.
[29]	MOU L C , GHAMISI P , ZHU X X . Unsupervised spectral–spatial feature learning via deep residual conv-deconv network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(1): 391-406.
[30]	MA X R , FU A Y , WANG J ,et al. Hyperspectral image classification based on deep deconvolution network with skip architecture[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(8): 4781-4791.
[31]	LEE H , KWON H . Going deeper with contextual CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2017,26(10): 4843-4855.
[32]	GAO H M , YANG Y , LEI S ,et al. Multi-branch fusion network for hyperspectral image classification[J]. Knowledge-Based Systems, 2019167:11

层	内核		输出尺寸
层	内核	IP数据集	PU数据集	SA数据集
图像块	—	21像素×21像素,15层	21像素×21像素,15层	33像素×33像素,21层
卷积层1	5×5	21像素×21像素,16层	21像素×21像素,24层	33像素×33像素,16层
最大池化层	3×3	7像素×7像素,16层	7像素×7像素,24层	11像素×11像素,16层
转置卷积和上采样	3×3	21像素×21像素,16层	21像素×21像素,24层	33像素×33像素,16层
结合层	—	21像素×21像素,32层	21像素×21像素,48层	33像素×33像素,32层
卷积层2	5×5	21像素×21像素,64层	21像素×21像素,96层	33像素×33像素,64层
1×1卷积	1×1	21像素×21像素,64层	21像素×21像素,96层	33像素×33像素,64层
融合层	—	21像素×21像素,64层	21像素×21像素,96层	33像素×33像素,64层
全连接1	Relu		128
全连接2	Relu		64
训练参数	—	3 924 928	5 922 401	9 074 528
双边融合块个数	—	4	3	2

超链接	上采样	转置卷积	OA	AA	KA
×	×	×	97.78%	91.19%	97.46%
×	×	√	98.15%	95.53%	97.89%
×	√	×	98.29%	96.61%	98.05%
√	×	×	97.83%	95.46%	97.53%
×	√	√	98.37%	96.82%	98.14%
√	√	×	98.39%	96.58%	98.17%
√	×	√	98.35%	95.33%	98.12%
√	√	√	98.45%	96.12%	98.23%

训练样本占比	KA	OA	AA
1%	59.53%	64.97%	46.41%
3%	84.88%	86.81%	73.92%
5%	94.03%	94.78%	85.37%
7%	96.70%	97.11%	91.97%
10%	98.23%	98.45%	96.12%

训练样本占比	KA	OA	AA
0.5%	86.57%	89.99%	81.53%
1%	95.10%	96.41%	92.82%
2%	98.33%	98.74%	97.74%
3%	99.09%	99.31%	98.56%
5%	99.53%	99.65%	99.14%

训练样本占比	KA	OA	AA
0.1%	80.32%	82.46%	76.41%
0.5%	98.56%	98.71%	98.64%
1%	99.68%	99.71%	99.62%
2%	99.86%	99.87%	99.80%
3%	99.91%	99.92%	99.91%

基于CNN的双边融合网络在高光谱图像分类中的应用

Application of bilateral fusion model based on CNN in hyperspectral image classification

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 32

相关文章 15

Metrics

推荐阅读 0

类别	RPCA-CNN模型	FCNN模型	DCNN模型	SSRN模型	DRN模型	DFBN模型
Alfalfa	34.88%	90.12%	43.66%	93.41%	90.37%	96.10%
Corn-notill	60.91%	93.11%	89.91%	97.59%	94.30%	95.54%
Corn-mintill	57.19%	97.03%	93.68%	98.71%	96.14%	99.35%
Corn	54.32%	93.92%	82.86%	91.22%	85.12%	98.99%
Grass-pasture	92.08%	96.34%	97.31%	99.79%	98.31%	98.77%
Grass-trees	97.63%	97.60%	99.09%	99.70%	98.86%	99.06%
Grass-pasture-mowed	79.20%	99.00%	91.20%	98.60%	93.80%	96.20%
Hay-windrowed	99.93%	99.30%	99.91%	99.84%	99.95%	99.97%
Oats	57.22%	82.78%	69.17%	95.28%	78.06%	73.89%
Soybean-notill	76.05%	97.49%	93.69%	99.25%	98.13%	98.63%
Soybean-mintill	63.95%	98.28%	94.14%	96.30%	99.04%	99.51%
Soybean-clean	48.16%	91.83%	91.22%	97.21%	92.36%	96.47%
Wheat	99.30%	96.32%	99.76%	99.78%	98.57%	97.49%
Woods	94.79%	99.78%	98.78%	99.73%	99.73%	99.93%
Buildings-Grass-Trees-Drives	77.59%	93.78%	85.46%	95.26%	96.71%	98.90%
Stone-Steel-Towers	99.82%	87.20%	96.13%	98.75%	89.17%	89.11%

分类模型	KA	OA	AA
RPCA-CNN模型	70.08%	73.65%	74.56%
FCNN模型	96.15%	96.63%	94.62%
DCNN模型	93.05%	93.91%	89.12%
SSRN模型	97.64%	97.93%	97.53%
DRN模型	96.75%	97.15%	94.29%
DFBN模型	98.23%	98.45%	96.12%

分类模型	KA	OA	AA
RPCA-CNN模型	75.39%	80.94%	85.15%
FCNN模型	84.91%	88.74%	78.32%
DCNN模型	92.40%	94.30%	89.42%
SSRN模型	97.88%	98.40%	97.60%
DRN模型	96.30%	97.21%	95.16%
DFBN模型	98.33%	98.74%	97.74%

分类模型	KA	OA	AA
RPCA-CNN模型	61.45%	64.70%	68.48%
FCNN模型	77.87%	80.21%	81.08%
DCNN模型	84.00%	85.60%	86.76%
SSRN模型	88.64%	89.79%	92.66%
DRN模型	94.87%	95.39%	95.15%
DFBN模型	98.56%	98.71%	98.64%

[1]	李昂, 陈建新, 魏昕, 周亮. 面向6G的跨模态信号重建技术[J]. 通信学报, 2022, 43(6): 28-40.
[2]	王晓丹, 李京泰, 宋亚飞. DDAC：面向卷积神经网络图像隐写分析模型的特征提取方法[J]. 通信学报, 2022, 43(5): 68-81.
[3]	廖育荣, 王海宁, 林存宝, 李阳, 方宇强, 倪淑燕. 基于深度学习的光学遥感图像目标检测研究进展[J]. 通信学报, 2022, 43(5): 190-203.
[4]	张帆, 黄赟, 方子茁, 郭威. 卷积神经网络的损失最小训练后参数量化方法[J]. 通信学报, 2022, 43(4): 114-122.
[5]	朱政宇, 侯庚旺, 黄崇文, 孙钢灿, 郝万明, 梁静. 基于并行CNN的RIS辅助D2D保密通信系统资源分配算法[J]. 通信学报, 2022, 43(3): 172-179.
[6]	安泽亮, 张天骐, 马宝泽, 邓盼, 徐雨晴. 基于一维CNN的多入多出OSTBC信号协作调制识别[J]. 通信学报, 2021, 42(7): 84-94.
[7]	王洪雁, 张莉彬, 陈国强, 汪祖民, 管志远. 结合粒子滤波及度量学习的目标跟踪方法[J]. 通信学报, 2021, 42(5): 98-110.
[8]	司念文, 张文林, 屈丹, 常禾雨, 李盛祥, 牛铜. 基于对抗补丁的可泛化的Grad-CAM攻击方法[J]. 通信学报, 2021, 42(3): 23-35.
[9]	高红民, 曹雪莹, 陈忠昊, 花再军, 李臣明, 陈月. 基于多尺度近端特征拼接网络的高光谱图像分类方法[J]. 通信学报, 2021, 42(2): 92-102.
[10]	杨军,党吉圣. 基于上下文注意力CNN的三维点云语义分割[J]. 通信学报, 2020, 41(7): 195-203.
[11]	张猛,孙昊良,杨鹏. 基于改进卷积神经网络识别DNS隐蔽信道[J]. 通信学报, 2020, 41(1): 169-179.
[12]	周鑫,何晓新,郑昌文. 基于图像深度学习的无线电信号识别[J]. 通信学报, 2019, 40(7): 114-125.
[13]	查雄,彭华,秦鑫,李广,李天昀. 基于多端卷积神经网络的调制识别方法[J]. 通信学报, 2019, 40(11): 30-37.
[14]	李佳,云晓春,李书豪,张永铮,谢江,方方. 基于混合结构深度神经网络的HTTP恶意流量检测方法[J]. 通信学报, 2019, 40(1): 24-33.
[15]	周胜利,金苍宏,吴礼发,洪征. 基于评分卡—随机森林的云计算用户公共安全信誉模型研究[J]. 通信学报, 2018, 39(5): 143-152.

类别	RPCA-CNN模型	FCNN模型	DCNN模型	SSRN模型	DRN模型	DFBN模型
Asphalt	70.04%	86.46%	96.79%	97.40%	97.42%	98.16%
Meadows	84.28%	98.41%	99.23%	99.89%	99.75%	99.86%
Gravel	88.17%	65.25%	67.98%	88.64%	89.02%	96.11%
Trees	92.97%	71.72%	97.03%	97.08%	95.81%	96.60%
Painted metal sheets	100.00%	84.11%	96.50%	99.91%	94.94%	97.22%
Bare Soil	75.27%	91.61%	92.10%	99.54%	99.73%	99.61%
Bitumen	95.69%	61.62%	67.64%	99.55%	94.93%	98.81%
Self-Blocking Bricks	60.09%	86.58%	88.08%	96.44%	88.13%	97.52%
Shadows	99.88%	59.14%	99.41%	99.98%	96.75%	95.78%

类别	RPCA-CNN模型	FCNN模型	DCNN模型	SSRN模型	DRN模型	DFBN模型
Brocoli_green_weeds_1	99.99%	57.71%	77.03%	95.48%	97.36%	99.45%
Brocoli_green_weeds_2	81.90%	78.83%	98.25%	97.97%	99.34%	99.96%
Fallow	22.50%	85.04%	82.33%	87.17%	94.87%	99.97%
Fallow_rough_plow	91.23%	81.69%	98.76%	97.45%	93.77%	98.34%
Fallow_smooth	59.10%	97.54%	92.29%	95.11%	93.54%	98.57%
Stubble	99.53%	93.73%	99.94%	99.90%	98.23%	99.81%
Celery	93.05%	77.67%	98.40%	98.79%	98.16%	99.48%
Grapes_untrained	36.76%	78.56%	69.84%	80.93%	95.05%	96.78%
Soil_vinyard_develop	72.10%	97.90%	99.12%	99.55%	99.70%	100.00%
Corn_senesced_green_weeds	73.37%	98.07%	91.87%	93.60%	97.35%	99.77%
Lettuce_romaine_4wk	60.86%	84.15%	25.40%	86.31%	92.99%	98.25%
Lettuce_romaine_5wk	22.66%	96.44%	98.61%	91.50%	89.96%	99.71%
Lettuce_romaine_6wk	41.34%	84.12%	98.98%	98.09%	89.46%	93.87%
Lettuce_romaine_7wk	97.60%	91.01%	88.94%	93.96%	92.79%	95.75%
Vinyard_untrained	63.91%	55.72%	74.50%	72.31%	89.33%	98.62%
Vinyard_vertical_trellis	86.25%	39.11%	93.87%	94.39%	97.75%	99.87%