基于深度学习的光学遥感图像目标检测研究进展

doi:10.11959/j.issn.1000-436x.2022071

通信学报 ›› 2022, Vol. 43 ›› Issue (5): 190-203.doi: 10.11959/j.issn.1000-436x.2022071

基于深度学习的光学遥感图像目标检测研究进展

廖育荣¹, 王海宁², 林存宝¹, 李阳², 方宇强¹, 倪淑燕¹

¹ 航天工程大学电子与光学工程系，北京 101416
² 航天工程大学研究生院，北京 101416

修回日期:2022-03-23 出版日期:2022-05-25 发布日期:2022-05-01
作者简介:廖育荣（1972- ），男，四川德阳人，博士，航天工程大学研究员，主要研究方向为空间智能信息处理、航天测控通信、卫星信号处理技术等
王海宁（1998- ），男，河南南阳人，航天工程大学硕士生，主要研究方向为遥感图像处理、深度学习、目标检测
林存宝（1987- ），男，吉林延边人，博士，航天工程大学助理研究员，主要研究方向为空间光学载荷、空间智能信息处理
李阳（1993- ），女，辽宁大连人，航天工程大学博士生，主要研究方向为遥感图像处理、目标检测
方宇强（1984- ），男，四川乐山人，博士，航天工程大学副教授，主要研究方向为计算机视觉、机器学习、深度学习
倪淑燕（1981- ），女，河北清河人，博士，航天工程大学副教授，主要研究方向为数字图像处理、空间智能信息处理等
基金资助:
国家自然科学基金资助项目(61805283);国家自然科学基金资助项目(61805284);国家自然科学基金资助项目(61906213)

Research progress of deep learning-based object detection of optical remote sensing image

Yurong LIAO¹, Haining WANG², Cunbao LIN¹, Yang LI², Yuqiang FANG¹, Shuyan NI¹

¹ Department of Electronic and Optical Engineering, Space Engineering University, Beijing 101416, China
² Department of Graduate Management, Space Engineering University, Beijing 101416, China

Revised:2022-03-23 Online:2022-05-25 Published:2022-05-01
Supported by:
The National Natural Science Foundation of China(61805283);The National Natural Science Foundation of China(61805284);The National Natural Science Foundation of China(61906213)

摘要/Abstract

摘要：

目标检测是光学遥感图像解译中的核心问题，在情报侦察、目标监视、灾害救援等领域均具有广泛应用。首先，结合深度学习光学遥感图像目标检测算法研究进展，对基于候选区域和回归分析的两类遥感目标检测算法进行了综述。其次，针对旋转目标、小目标、多尺度、密集目标四类常见特定任务场景目标检测算法改进进行了总结。再次，结合常用遥感图像数据集对不同算法性能进行了对比分析。最后，对未来遥感图像目标检测研究值得关注的问题进行了展望，为后续相关研究提供思路。

关键词: 光学遥感图像, 目标检测, 深度学习, 卷积神经网络

Abstract:

Object detection is the core issue in the interpretation of optical remote sensing images, and it is widely used in fields such as intelligence reconnaissance, target monitoring, and disaster rescue.Firstly, combined with the research progress of deep learning optical remote sensing image object detection algorithms, the two types of algorithms based on candidate regions and regression analysis were reviewed.Secondly, the improvement of object detection algorithms for four types of common task-specific scenes were summarized, including rotating objects, small objects, multi-scales, and dense objects.Then, combined with commonly used remote sensing image data sets, the performance of different algorithms was compared and analyzed.Finally, the issues worthy of attention in remote sensing image object detection in the future were prospected, and ideas for follow-up related research were provided.

Key words: optical remote sensing image, object detection, deep learning, convolutional neural network

中图分类号:

TP391.41

廖育荣, 王海宁, 林存宝, 李阳, 方宇强, 倪淑燕. 基于深度学习的光学遥感图像目标检测研究进展[J]. 通信学报, 2022, 43(5): 190-203.

Yurong LIAO, Haining WANG, Cunbao LIN, Yang LI, Yuqiang FANG, Shuyan NI. Research progress of deep learning-based object detection of optical remote sensing image[J]. Journal on Communications, 2022, 43(5): 190-203.

图/表 8

图1

表1

图2

表2

图3

表3

表4

表5

参考文献 118

[1]	CHENG G , HAN J W . A survey on object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016,117: 11-28.
[2]	WANG Z , BOVIK A C , SHEIKH H R ,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004,13(4): 1-14.
[3]	WU W . Quantized Gromov-Hausdorff distance[J]. Journal of Functional Analysis, 2006,238: 58-98.
[4]	HAREL J , KOCH C , PERONA P . Graph-based visual saliency[J]. Advances in Neural Information Processing Systems, 2006,19: 545-552.
[5]	SEDAGHAT A , EBADI H . Remote sensing image matching based on adaptive binning SIFT descriptor[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015,53(10): 5283-5293.
[6]	DALAL N , TRIGGS B . Histograms of oriented gradients for human detection[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2005: 886-893.
[7]	GóMEZ C L , TUIA D , MOSER G ,et al. Multimodal classification of remote sensing images:a review and future directions[J]. Proceedings of the IEEE, 2015,103(9): 1560-1584.
[8]	ZHU X X , TUIA D , MOU L C ,et al. Deep learning in remote sensing:a comprehensive review and list of resources[J]. IEEE geoscience and remote sensing magazine, 2017,5(4): 8-36.
[9]	罗会兰, 陈鸿坤 . 基于深度学习的目标检测研究综述[J]. 电子学报, 2020,48(6): 1230-1239.
	LUO H L , CHEN H K . Survey of object detection based on deep learning[J]. ACTA ELECTRONICA SINICA, 2021,48(6): 1230-1239.
[10]	LI Y S , ZHANG Y J , HUANG X ,et al. Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018,146: 182-196.
[11]	XUE B , TONG N N . DIOD:fast and efficient weakly semi-supervised deep complex ISAR object detection[J]. IEEE Transactions on Cybernetics, 2019,49(11): 3991-4003.
[12]	YU D W , JI S P . A new spatial-oriented object detection framework for remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60: 1-16.
[13]	周勇, 陈思霖, 赵佳 ,等. 基于弱语义注意力的遥感图像可解释目标检测[J]. 电子学报, 2021,49(4): 679-689.
	ZHOU Y , CHEN S L , ZHAO J ,et al. Weakly semantic based attention network for interpretable object detection in remote sensing imagery[J]. ACTA ELECTRONICA SINICA, 2021,49(4): 679-689.
[14]	ZHANG Y J , SHENG W G , JIA J F ,et al. Priority branches for ship detection in optical remote sensing[J]. Remote Sensing, 2020,12(7): 1196.
[15]	LI K , WAN G , CHENG G ,et al. Object detection in optical remote sensing images:a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020,159: 296-307.
[16]	聂光涛, 黄华 . 光学遥感图像目标检测算法综述[J]. 自动化学报, 2021,47(8): 1749-1768.
	NIE G T , HUANG H . A survey of object detection in optical remote sensing images[J]. ACTA AUTOMATICA SINICA, 2021,47(8): 1749-1768.
[17]	宋志娜, 眭海刚, 李永成 . 高分辨率可见光遥感图像舰船目标检测综述[J]. 武汉大学学报(信息科学版), 2021:doi.org/10.13203/j.whugis20200481.
	SONG Z N , SUI H G , LI Y C . A survey on ship detection technology in high-resolution optical remote sensing images[J]. Geomatics and Information Science of Wuhan University, 2021:doi.org/10.13203/j.whugis20200481.
[18]	GIRSHICK R , DONAHUE J , DARRELL T ,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 580-587.
[19]	GIRSHICK R , . Fast R-CNN[C]// Proceedings of IEEE Conference on Computer Vision. Piscataway:IEEE Press, 2015: 1440-1448.
[20]	REN S Q , HE K M , GIRSHICK R ,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149.
[21]	XU C Y , LI C Z , CUI Z ,et al. Hierarchical semantic propagation for object detection in remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(6): 4353-4364.
[22]	FU K , CHANG Z H , ZHANG Y ,et al. Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020,161: 294-308.
[23]	CHEN C Y , GONG W G , CHEN Y L ,et al. Object detection in remote sensing images based on a scene-contextual feature pyramid network[J]. Remote Sensing, 2019,11(3): 339.
[24]	YAN D C , LI G Q , LI X Q ,et al. An improved faster R-CNN method to detect tailings ponds from high-resolution remote sensing images[J]. Remote Sensing, 2021,13(11): 2052.
[25]	ZHANG G J , LU S J , ZHANG W . CAD-Net:a context-aware detection network for objects in remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(12): 10015-10024.
[26]	LI Q P , MOU L C , LIU Q J ,et al. HSF-Net:multiscale deep feature embedding for ship detection in optical remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(12): 7147-7161.
[27]	赵彤洲, 杨成万, 刘威 . 非局部特征增强的遥感图像目标检测方法[J]. 华中科技大学学报(自然科学版), 2021:doi.org/10.13245/j.hust.210909.
	ZHAO T Z , YANG C W , LIU W . Remote sensing image object detection method based on non-local feature enhancement[J]. Journal of Huazhong University of Science ＆ Technology.(Natural Science Edition), 2021:doi.org/10.13245/j.hust.210909.
[28]	CHENG G , HAN J W , ZHOU P C ,et al. Learning rotation-invariant and fisher discriminative convolutional neural networks for object detection[J]. IEEE Transactions on Image Processing, 2019,28(1): 265-278.
[29]	HAN X B , ZHONG Y F , ZHANG L P . An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery[J]. Remote Sensing, 2017,9(7): 666.
[30]	ZHONG Y F , HAN X B , ZHANG L P . Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018,138: 281-294.
[31]	LONG Y , GONG Y P , XIAO Z F ,et al. Accurate object localization in remote sensing images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(5): 2486-2498.
[32]	LI K , CHENG G , BU S H ,et al. Rotation-insensitive and context-augmented object detection in remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(4): 2337-2348.
[33]	DONG R C , XU D Z , ZHAO J ,et al. Sig-NMS-based faster R-CNN combining transfer learning for small target detection in VHR optical remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(11): 8534-8545.
[34]	仲伟峰, 郭峰, 向世明 ,等. 旋转矩形区域的遥感图像舰船目标检测模型[J]. 计算机辅助设计与图形学学报, 2019,31(11): 1935-1945.
	ZHONG W F , GUO F , XIANG S M ,et al. Ship detection in remote sensing based with rotated rectangular region[J]. Journal of Computer-Aided Design ＆ Computer Graphics, 2019,31(11): 1935-1945.
[35]	REN Y , ZHU C R , XIAO S P . Deformable Faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images[J]. Remote Sensing, 2018,10(9): 1470.
[36]	DING P , ZHANG Y , DENG W J ,et al. A light and faster regional convolutional neural network for object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018,141: 208-218.
[37]	ZHANG S M , WU R Z , XU K Y ,et al. R-CNN-based ship detection from high resolution remote sensing imagery[J]. Remote Sensing, 2019,11(6): 631.
[38]	YAN J Q , WANG H Q , YAN M L ,et al. IoU-adaptive deformable R-CNN:make full use of IoU for multi-class object detection in remote sensing imagery[J]. Remote Sensing, 2019,11(3): 286.
[39]	HE K M , GKIOXARI G , DOLLáR P ,et al. Mask R-CNN[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 2980-2988.
[40]	WU Q F , FENG D Q , CAO C Q ,et al. Improved Mask R-CNN for aircraft detection in remote sensing images[J]. Sensors, 2021,21(8): 2618.
[41]	DING J , XUE N , LONG Y ,et al. Learning RoI transformer for oriented object detection in aerial images[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2019: 2844-2853.
[42]	ZHU Y X , DU J , WU X Q . Adaptive period embedding for representing oriented objects in aerial images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(10): 7247-7257.
[43]	XU Z Z , XU X , WANG L ,et al. Deformable ConvNet with aspect ratio constrained NMS for object detection in remote sensing imagery[J]. Remote Sensing, 2017,9(12): 1312.
[44]	CHEN H B , JIANG S , HE G H ,et al. TEANS:a target enhancement and attenuated nonmaximum suppression object detector for remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2021,18(4): 632-636.
[45]	REDMON J , DIVVALA S , GIRSHICK R ,et al. You only look once:unified,real-time object detection[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 779-788.
[46]	LIU W , ANGUELOV D , ERHAN D ,et al. SSD:single shot multibox detector[C]// Proceedings of European Conference on Computer Vision. Berlin:Springer, 2016: 21-37.
[47]	REDMON J , FARHADI A . YOLO9000:better,faster,stronger[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 6517-6525.
[48]	REDMON J , FARHADI A . YOLOv3:an incremental improvement[J]. arXiv Preprint,arXiv:1804.02767, 2018.
[49]	BOCHKOVSKIY A , WANG C Y , LIAO H Y . YOLOv4:optimal speed and accuracy of object detection[J]. arXiv Preprint,arXiv:2004.10934, 2020.
[50]	JOCHER G , STOKEN A , BOROVEC J ,et al. Ultralytics/yolov5:v4.0[EB]. 2021.
[51]	GE Z , LIU S T , WANG F ,et al. YOLOX:exceeding YOLO series in 2021[J]. arXiv Preprint,arXiv:2107.08430, 2021.
[52]	MA H J , LIU Y L , REN Y H ,et al. Detection of collapsed buildings in post-earthquake remote sensing images based on the improved YOLOv3[J]. Remote Sensing, 2019,12(1): 44.
[53]	CHANG Y L , ANAGAW A , CHANG L N ,et al. Ship detection based on YOLOv2 for SAR imagery[J]. Remote Sensing, 2019,11(7): 786.
[54]	WANG C S , WANG Q , WU H R ,et al. Low-altitude remote sensing opium poppy image detection based on modified YOLOv3[J]. Remote Sensing, 2021,13(11): 2130.
[55]	XU D Q , WU Y Q . FE-YOLO:a feature enhancement network for remote sensing target detection[J]. Remote Sensing, 2021,13(7): 1311.
[56]	YANG X , YAN J C , FENG Z M ,et al. R³Det:refined single-stage detector with feature refinement for rotating object[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2021: 3163-3171.
[57]	LIU W C , MA L , WANG J ,et al. Detection of multiclass objects in optical remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2019,16(5): 791-795.
[58]	XU D Q , WU Y Q . MRFF-YOLO:a multi-receptive fields fusion network for remote sensing target detection[J]. Remote Sensing, 2020,12(19): 3118.
[59]	HU J M , ZHI X Y , SHI T J ,et al. PAG-YOLO:a portable attention-guided YOLO network for small ship detection[J]. Remote Sensing, 2021,13(16): 3059.
[60]	CHEN L Q , SHI W X , DENG D X . Improved YOLOv3 based on attention mechanism for fast and accurate ship detection in optical remote sensing images[J]. Remote Sensing, 2021,13(4): 660.
[61]	XU D Q , WU Y Q . Improved YOLO-V3 with DenseNet for multi-scale remote sensing target detection[J]. Sensors, 2020,20(15): 4276.
[62]	WU K J , BAI C S , WANG D C ,et al. Improved object detection algorithm of YOLOv3 remote sensing image[J]. IEEE Access, 2021,9: 113889-113900.
[63]	PHAM M T , COURTRAI L , FRIGUET C ,et al. YOLO-fine:one-stage detector of small objects under various backgrounds in remote sensing images[J]. Remote Sensing, 2020,12(15): 2501.
[64]	VAN ETTEN A . You only look twice:rapid multi-scale object detection in satellite imagery[J]. arXiv Preprint,arXiv:1805.09512, 2018.
[65]	HONG Z H , YANG T , TONG X H ,et al. Multi-scale ship detection from SAR and optical imagery via a more accurate YOLOv3[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021,14: 6083-6101.
[66]	SHARMA M , DHANARAJ M , KARNAM S ,et al. YOLOrs:object detection in multimodal remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020,14: 1497-1508.
[67]	LU X C , JI J , XING Z Q ,et al. Attention and feature fusion SSD for remote sensing object detection[J]. IEEE Transactions on Instrumentation and Measurement, 2021,70: 1-9.
[68]	QU J S , SU C , ZHANG Z W ,et al. Dilated convolution and feature fusion SSD network for small object detection in remote sensing images[J]. IEEE Access, 2020,8: 82832-82843.
[69]	YIN R H , ZHAO W , FAN X D ,et al. AF-SSD:an accurate and fast single shot detector for high spatial remote sensing imagery[J]. Sensors, 2020,20(22): 6530.
[70]	BAO S Z , ZHONG X , ZHU R F ,et al. Single shot anchor refinement network for oriented object detection in optical remote sensing imagery[J]. IEEE Access, 2019,7: 87150-87161.
[71]	TANG T Y , ZHOU S L , DENG Z P ,et al. Arbitrary-oriented vehicle detection in aerial imagery with single convolutional neural networks[J]. Remote Sensing, 2017,9(11): 1170.
[72]	LIU L , PAN Z X , LEI B . Learning a rotation invariant detector with rotatable bounding box[J]. arXiv Preprint,arXiv:1711.09405, 2017.
[73]	郭智, 宋萍, 张义 ,等. 基于深度卷积神经网络的遥感图像飞机目标检测方法[J]. 电子与信息学报, 2018,40(11): 2684-2690.
	GUO Z , SONG P , ZHANG Y ,et al. Aircraft detection method based on deep convolutional neural network for remote sensing images[J]. Journal of Electronics ＆ Information Technology, 2018,40(11): 2684-2690.
[74]	李晖晖, 周康鹏, 韩太初 . 基于CReLU和FPN改进的 SSD舰船目标检测[J]. 仪器仪表学报, 2020,41(4): 183-190.
	LI H H , ZHOU K P , HAN T C . Ship object detection based on SSD improved with CReLU and FPN[J]. Chinese Journal of Scientific Instrument, 2020,41(4): 183-190.
[75]	WANG P J , SUN X , DIAO W H ,et al. FMSSD:feature-merged single-shot detection for multiscale objects in large-scale remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(5): 3377-3390.
[76]	SCHILLING H , BULATOV D , NIESSNER R ,et al. Detection of vehicles in multisensor data via multibranch convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018,11(11): 4299-4316.
[77]	QIAN W , YANG X , PENG S L ,et al. Learning modulated loss for rotated object detection[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2021: 2458-2466.
[78]	XU Y C , FU M T , WANG Q M ,et al. Gliding vertex on the horizontal bounding box for multi-oriented object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(4): 1452-1459.
[79]	YANG X , YANG J R , YAN J C ,et al. SCRDet:towards more robust detection for small,cluttered and rotated objects[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2019: 8231-8240.
[80]	HOU L P , LU K , XUE J ,et al. Cascade detector with feature fusion for arbitrary-oriented objects in remote sensing images[C]// Proceedings of 2020 IEEE International Conference on Multimedia and Expo. Piscataway:IEEE Press, 2020: 1-6.
[81]	YANG X , YAN J C , HE T . Arbitrary-oriented object detection with circular smooth label[J]. arXiv Preprint,arXiv:2003.05597, 2020.
[82]	YANG X , HOU L P , ZHOU Y ,et al. Dense label encoding for boundary discontinuity free rotation detection[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2021: 15814-15824.
[83]	YANG X , YAN J C , MING Q ,et al. Rethinking rotated object detection with Gaussian Wasserstein distance loss[C]// Proceedings of International Conference on Machine Learning. New York:ACM Press, 2021: 11830-11841.
[84]	PAN X J , REN Y Q , SHENG K K ,et al. Dynamic refinement network for oriented and densely packed object detection[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 11204-11213.
[85]	WEI H R , ZHANG Y , CHANG Z H ,et al. Oriented objects as pairs of middle lines[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020,169: 268-279.
[86]	FENG P M , LIN Y T , GUAN J ,et al. TOSO:student’s-T distribution aided one-stage orientation target detection in remote sensing images[C]// Proceedings of 2020 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2020: 4057-4061.
[87]	LI Y Y , HUANG Q , PEI X ,et al. RADet:refine feature pyramid network and multi-layer attention network for arbitrary-oriented object detection of remote sensing images[J]. Remote Sensing, 2020,12(3): 389.
[88]	CHENG G , ZHOU P C , HAN J W . Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016,54(12): 7405-7415.
[89]	LI C Y , CONG R M , GUO C L ,et al. A parallel down-up fusion network for salient object detection in optical remote sensing images[J]. Neurocomputing, 2020,415: 411-420.
[90]	ZHANG Q J , CONG R M , LI C Y ,et al. Dense attention fluid network for salient object detection in optical remote sensing images[J]. IEEE Transactions on Image Processing, 2021,30: 1305-1317.
[91]	DENG Z P , SUN H , ZHOU S L ,et al. Multi-scale object detection in remote sensing imagery with convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018,145: 3-22.
[92]	DONG Z P , WANG M , WANG Y L ,et al. Object detection in high resolution remote sensing imagery based on convolutional neural networks with suitable object scale features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(3): 2104-2114.
[93]	WANG C , BAI X , WANG S ,et al. Multiscale visual attention networks for object detection in VHR remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2019,16(2): 310-314.
[94]	LI C Y , LUO B , HONG H L ,et al. Object detection based on global-local saliency constraint in aerial images[J]. Remote Sensing, 2020,12(9): 1435.
[95]	YU Y T , GUAN H Y , LI D L ,et al. Orientation guided anchoring for geospatial object detection from remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020,160: 67-82.
[96]	HUA X , WANG X Q , RUI T ,et al. A fast self-attention cascaded network for object detection in large scene remote sensing images[J]. Applied Soft Computing, 2020,94:106495.
[97]	ZHENG Z , ZHONG Y F , MA A L ,et al. HyNet:hyper-scale object detection network framework for multiple spatial resolution remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020,166: 1-14.
[98]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Annual Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2017: 5998-6008.
[99]	ZHU X K , LYU S C , WANG X ,et al. TPH-YOLOv5:improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Piscataway:IEEE Press, 2021: 2778-2788.
[100]	QING Y H , LIU W Y , FENG L Y ,et al. Improved YOLO network for free-angle remote sensing target detection[J]. Remote Sensing, 2021,13(11): 2171.
[101]	HEITZ G , KOLLER D . Learning spatial context:using stuff to find things[C]// Lecture Notes in Computer Science. Berlin:Springer, 2008: 30-43.
[102]	FRANKLIN T , BRIAN C , CRAIG P ,et al. Overhead imagery research data set - an annotated data library ＆ tools to aid in the development of computer vision algorithms[C]// Proceedings of 2009 IEEE Applied Imagery Pattern Recognition Workshop (AIPR 2009). Piscataway:IEEE Press, 2009: 1-8.
[103]	MAGGIORI E , TARABALKA Y , CHARPIAT G ,et al. Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark[C]// Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium. Piscataway:IEEE Press, 2017: 3226-3229.
[104]	ZHU H G , CHEN X G , DAI W Q ,et al. Orientation robust object detection in aerial images using deep convolutional neural network[C]// Proceedings of 2015 IEEE International Conference on Image Processing. Piscataway:IEEE Press, 2015: 3735-3739.
[105]	CHENG G , HAN J W , ZHOU P C ,et al. Multi-class geospatial object detection and geographic image classification based on collection of part detectors[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014,98: 119-132.
[106]	RAZAKARIVONY S , JURIE F . Vehicle detection in aerial imagery:a small target detection benchmark[J]. Journal of Visual Communication and Image Representation, 2016,34: 187-203.
[107]	LIU Z K , YUAN L , WENG L B ,et al. A high resolution optical satellite image dataset for ship recognition and some new baselines[C]// Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods. Francisco:SciTePress, 2017: 324-331.
[108]	LIU K , MATTYUS G . Fast multiclass vehicle detection on aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2015,12(9): 1938-1942.
[109]	ZHANG Y L , YUAN Y , FENG Y C ,et al. Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(8): 5535-5548.
[110]	ZOU Z X , SHI Z W . Random access memories:a new paradigm for target detection in high resolution aerial remote sensing images[J]. IEEE Transactions on Image Processing, 2018,27(3): 1100-1111.
[111]	YANG Y M . ITCVD dataset[EB]. 2018.
[112]	XIA G S , BAI X , DING J ,et al. DOTA:a large-scale dataset for object detection in aerial images[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 3974-3983.
[113]	SUN X , WANG P J , YAN Z Y ,et al. FAIR1M:a benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022,184: 116-130.
[114]	CHENG G , YAN B W , SHI P Z ,et al. Prototype-CNN for few-shot object detection in remote sensing images[C]// Proceedings of IEEE Transactions on Geoscience and Remote Sensing. Piscataway:IEEE Press, 2022: 1-10.
[115]	LI X , DENG J Y , FANG Y . Few shot object detection in remote sensing images[C]// Proceedings of IEEE Transactions on Geoscience and Remote Sensing. Piscataway:IEEE Press, 2021: 76-81.
[116]	CUI Y Y , HOU B , WU Q ,et al. Remote sensing object tracking with deep reinforcement learning under occlusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60: 1-13.
[117]	MERANER A , EBEL P , ZHU X X ,et al. Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020,166: 333-346.
[118]	DU Q , TANG B , XIE W Y ,et al. Parallel and distributed computing for anomaly detection from hyperspectral remote sensing imagery[J]. Proceedings of the IEEE, 2021,109(8): 1306-1319.

文献	算法亮点	适用范围
文献[21-22]	引入特征融合模块，在准确提取深层网络特征的同时简化计算空间复杂度
文献[23-25]	采用区域上下文信息增强目标特征与对应场景之间的联系
文献[26]	调整特征图尺度变换的方式，增强遥感目标检测的特征表征能力	针对一般遥感目标特征表示阶段改进
文献[27]	固定角度旋转进行数据增强，调整特征图尺度变换的方式
文献[28]	浅层卷积特征提取时使用正则化器
文献[29]	引入特征共享集成机制
文献[30]	提出位置敏感平衡策略
文献[31]	提出选择性搜索匹配策略
文献[32]	提出旋转不敏感RPN，有效提高了对遥感图像中多方向目标的检测效果
文献[33]	提出一种改进的NMS模块，以更好地进行目标区域筛选
文献[34]	提出一种兴趣区域特征金字塔池化模块	针对一般遥感目标感兴趣区域生成与处理阶段改进
文献[35]	提出一种ROI形变器
文献[36]	提出一种ROI排序方法
文献[37]	提出一种ROI旋转策略
文献[38]	将ROI Pooling层与不同交并比加权损失值相结合以提高目标区域定位精度
文献[40]	提出了一种结合掩模预测及语义分割算法，使ROI中保留更多目标空间特征向量
文献[41]	计算旋转IoU损失
文献[42]	计算联合交集IoU损失
文献[43]	设计了比例约束NMS算法，能够筛除遥感目标预选框中的错误结果	针对一般遥感目标提升定位精度改进
文献[44]	提出了衰减NMS算法，能够解决多种遥感目标不同尺度重叠的问题

文献	算法亮点	适用范围
文献[52-54]	在特殊应用背景下的成功探索
文献[55]	设计特征增强模块，能够提升目标的特征表达能力
文献[56]	设计特征精细化模块，能够实现目标特征的细致区分
文献[57]	结合空洞卷积，有效提升YOLO算法针对大目标的精度提升	对检测速度有更高要求的快速遥感目标检测场景
文献[58]	引入多感受域算法，提高了目标特征提取能力
文献[59-60]	引入注意力机制实现精度与速度间的最佳权衡
文献[61]	采取密集连接网络以增强目标特征提取能力
文献[62-66]	分别针对小目标、多尺度、多模态等专项遥感检测场景定制化改进
文献[67]	引入空间和通道注意力模块以筛选融合特征信息
文献[68]	结合空洞卷积与特征融合策略，能够增强浅层特征的语义信息
文献[69]	设计新的编码-解码模块，结合空间和通道注意力级联结构
文献[70]	采取连续两次回归的策略，使检测结果更加精确	对检测速度与精度间有均衡要求的一般遥感目标检测场景
文献[71]	通过定向边界框生成角度修正参数
文献[72]	通过旋转边界框学习遥感目标的正确角度方向
文献[73]	利用密集连接网络提取特征
文献[74]	结合激活函数CReLU和特征金字塔网络，能够提升网络的浅层特征的利用率
文献[75]	利用浅层信息并增大小目标训练权重以提升特征传递效率

文献	算法亮点	适用范围
文献[76]	随机角度旋转以进行数据增强
文献[77-78]	有序四边形表示法学习矩形各边的偏移
文献[79]	结合多层特征与锚定采样的特征融合网络
文献[80]	监督空间注意力机制与多步检测子网络级联相结合
文献[81-82]	将旋转角度回归转化为分类问题并限制分类结果的范围
文献[83]	2个任意方向高斯分布的Wasserstein距离作为旋转IoU损失	针对遥感旋转目标的改进
文献[84]	提出自适应调整的神经元感受野模块
文献[85]	提出目标中线以进行旋转角度回归
文献[86]	提出学生T-分布将分类任务与旋转角度回归相关联
文献[87]	提出掩模分支预测下获取目标旋转边界框
文献[88]	添加正则化项约束训练样本在前后的特征表示
文献[89]	减小网络降采样的倍数，并将浅层与深层特征融合拼接	针对遥感小目标的改进
文献[90]	在浅层特征图中聚合上下文信息，并增加小目标训练权重
文献[91]	构建包含不同尺度信息的特征融合架构
文献[92]	寻找合适的ROI尺寸进行多尺度特征映射与融合
文献[93]	将视觉注意力网络引入多尺度信息融合模块
文献[94]	结合全局注意力模块将多尺度特征图输入显著性金字塔	针对遥感多尺度目标的改进
文献[95]	提出定向引导的锚点框生成网络以生成合适的锚点框
文献[96]	利用特征金字塔进行不同尺度目标信息的提取
文献[97]	使用超尺度块作为核心结构构建超尺度目标检测框架
文献[99]	利用Transformer模型的预测主体提升多尺度预测能力
文献[100]	基于圆形平滑标签设计损失函数	针对遥感密集目标的改进

数据集	发布者及内容描述	目标类别数/个	图像数/幅
TAS ^[101]	由Stanford大学发布的车辆目标数据集	1	30
OIRDS ^[102]	由Raytheon公司发布的车辆目标数据集	5	900
SZTAKI ^[103]	由Mta Sztaki发布的旋转建筑物目标数据集	1	9
UCAS-AOD ^[104]	由中国科学院发布的车辆和飞机目标数据集，以及背景负样本	2	976
NWPU VHR-10 ^[105]	由西北工业大学发布的飞机、舰船、油罐、棒球场、网球场、篮球场等目标数据集	10	1 510
VEDAI ^[106]	由Caen大学发布的车辆目标数据集	9	1 210
HRSC2016 ^[107]	由西北工业大学发布的舰船目标数据集	1	1 061
DLR3k ^[108]	由German Aerospace Center发布的车辆目标数据集	7	20
RSOD ^[31]	由武汉大学发布的飞机、油箱、运动场和立交桥目标数据集	4	976
TGRS-HRRSD ^[109]	由中国科学院发布的舰船、桥梁、田径场、油罐、篮球场、网球场等目标数据集	13	21 761
LEVIR ^[110]	由北京航空航天大学发布的飞机、舰船、油罐目标数据集	3	22 000
ITCVD ^[111]	由Twente大学发布的车辆目标数据集	1	135
DIOR ^[15]	由西北工业大学发布的飞机、机场、篮球场、桥梁、烟囱、水坝等目标数据集	20	23 463
DOTA ^[112]	由武汉大学发布的舰船、游泳池、田径场、港口、直升机、足球场等目标数据集	16	2 806
FAIR1M ^[113]	由中国科学院等发布的飞机、舰船、车辆、球场、道路等5种大类别、37个细粒度类别的数据集，是目前全球规模最大的光学遥感图像细粒度目标检测识别数据集	37	15 000

数据集	算法	主干网络	改进功能				mAP
数据集	算法	主干网络	目标旋转	微小目标	多尺度	目标密集	mAP
	GWD^[83]	ResNet152	√			√	80.23
	HSP^[21]	ResNet101	√		√		78.01
	NLFE^[27]	ResNet50	√				77.46
	DCL^[82]	ResNet152	√	√			77.37
	R³Det^[56]	ResNet152	√			√	76.47
	CLS^[81]	ResNet152	√			√	76.24
	APERO^[42]	ResNet101	√	√			75.75
	FFA^[22]	ResNet101	√		√		75.70
	SCFPN^[23]	ResNet101			√		75.22
DOTA数据集	Gliding vertex^[78]	ResNet101	√				75.02
	RSDet^[77]	ResNet152	√			√	74.10
	DR-Net^[84]	CenterNet	√				73.23
	GLS-Net^[94]	ResNet101	√		√		72.96
	IoU-Adaptive^[38]	ResNet101		√	√		72.72
	FMSSD^[75]	VGG16		√	√		72.43
	CAD-Net^[25]	ResNet101	√				69.60
	RoITransformer^[41]	ResNet101	√		√		69.56
	TEANS^[44]	ResNet101		√			66.01
	AFF-SSD^[67]	ResNet50		√			52.60
	HyNet^[97]	ResNet50		√	√		98.52
	SCRDet^[79]	ResNet101	√	√		√	91.75
	MS-OPN AODN^[38]	GoogleNet		√	√	√	91.52
	CAD-Net^[25]	ResNet101	√				91.50
	FMSSD^[75]	VGG16		√	√		90.40
	AF-SSD^[69]	ShuffleNetV2		√			88.70
NWPU VHR-10数据集	RICA^[32]	ZFNet	√				87.12
	Deformable Faster RCNN^[35]	ResNet50					84.40
	Sig-NMS^[33]	VGG16		√			82.90
	PSB^[30]	ResNet101					81.20
	HSF-Net^[26]	VGG16		√	√		81.15
	Deformable R-FCN^[43]	ResNet101			√		79.10
	R-P-Faster R-CNN^[29]	VGG16			√		76.50
	DFSSD^[68]	VGG16		√			65.35

基于深度学习的光学遥感图像目标检测研究进展

Research progress of deep learning-based object detection of optical remote sensing image

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 118

相关文章 15

Metrics

推荐阅读 0

[1]	陈东昱, 陈华, 范丽敏, 付一方, 王舰. 基于深度学习的随机性检验策略研究[J]. 通信学报, 2023, 44(6): 23-33.
[2]	李荣鹏, 汪丙炎, 张宏纲, 赵志峰. 知识增强的语义通信接收端设计[J]. 通信学报, 2023, 44(6): 70-76.
[3]	马帅, 裴科, 祁华艳, 李航, 曹雯, 王洪梅, 熊海良, 李世银. 基于生成模型的地磁室内高精度定位算法研究[J]. 通信学报, 2023, 44(6): 211-222.
[4]	杨洁, 董标, 付雪, 王禹, 桂冠. 基于轻量化分布式学习的自动调制分类方法[J]. 通信学报, 2022, 43(7): 134-142.
[5]	李昂, 陈建新, 魏昕, 周亮. 面向6G的跨模态信号重建技术[J]. 通信学报, 2022, 43(6): 28-40.
[6]	杨秀璋, 彭国军, 李子川, 吕杨琦, 刘思德, 李晨光. 基于Bert和BiLSTM-CRF的APT攻击实体识别及对齐研究[J]. 通信学报, 2022, 43(6): 58-70.
[7]	王晓丹, 李京泰, 宋亚飞. DDAC：面向卷积神经网络图像隐写分析模型的特征提取方法[J]. 通信学报, 2022, 43(5): 68-81.
[8]	廖勇, 王世义. 高速移动环境下基于RM-Net的大规模MIMO CSI反馈算法[J]. 通信学报, 2022, 43(5): 166-176.
[9]	张帆, 黄赟, 方子茁, 郭威. 卷积神经网络的损失最小训练后参数量化方法[J]. 通信学报, 2022, 43(4): 114-122.
[10]	赵增华, 童跃凡, 崔佳洋. 基于域自适应的Wi-Fi指纹设备无关室内定位模型[J]. 通信学报, 2022, 43(4): 143-153.
[11]	朱政宇, 侯庚旺, 黄崇文, 孙钢灿, 郝万明, 梁静. 基于并行CNN的RIS辅助D2D保密通信系统资源分配算法[J]. 通信学报, 2022, 43(3): 172-179.
[12]	廖勇, 程港, 李玉杰. 基于深度展开的大规模MIMO系统CSI反馈算法[J]. 通信学报, 2022, 43(12): 77-88.
[13]	段雪源, 付钰, 王坤, 李彬. 基于简单统计特征的LDoS攻击检测方法[J]. 通信学报, 2022, 43(11): 53-64.
[14]	霍俊彦, 邱瑞鹏, 马彦卓, 杨付正. 基于最邻近帧质量增强的视频编码参考帧列表优化算法[J]. 通信学报, 2022, 43(11): 136-147.
[15]	康海燕, 冀源蕊. 基于本地化差分隐私的联邦学习方法研究[J]. 通信学报, 2022, 43(10): 94-105.