基于显著性信息的Fit CutMix数据增强算法在医学影像上的应用

doi:10.11959/j.issn.2096-6652.202307

智能科学与技术学报 ›› 2023, Vol. 5 ›› Issue (1): 58-68.doi: 10.11959/j.issn.2096-6652.202307

基于显著性信息的Fit CutMix数据增强算法在医学影像上的应用

罗欣欢¹^,², 王奕璇¹^,², 李炜¹^,², 陈曦¹^,²

¹ 华中科技大学人工智能与自动化学院，湖北武汉 430074
² 图像信息处理与智能控制教育部重点实验室，湖北武汉 430074

修回日期:2023-03-03 出版日期:2023-03-15 发布日期:2023-03-01
作者简介:罗欣欢（1998- ），女，华中科技大学人工智能与自动化学院硕士生，主要研究方向为基于数据增强的阿尔茨海默病辅助诊断
王奕璇（1998- ），男，华中科技大学人工智能与自动化学院硕士生，主要研究方向为基于深度学习的阿尔茨海默病早期诊断与转换预测
李炜（1975- ），女，华中科技大学人工智能与自动化学院教授，主要研究方向为神经信号分析、神经可塑性、脑网络分析和神经解码
陈曦（1975- ），男，华中科技大学人工智能与自动化学院教授，主要研究方向为信号处理与分析、系统建模与仿真、复杂网络等
基金资助:
国家自然科学基金资助项目(61473131)

Application of Fit CutMix data augmentation algorithm based on saliency information in medical images

Xinhuan LUO¹^,², Yixuan WANG¹^,², Wei LI¹^,², Xi CHEN¹^,²

¹ School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
² Key Laboratory of Image Processing and Intelligent Control, Ministry of Education, Huazhong University of Science and Technology, Wuhan 430074, China

Revised:2023-03-03 Online:2023-03-15 Published:2023-03-01
Supported by:
The National Natural Science Foundation of China(61473131)

摘要/Abstract

摘要：

深度卷积神经网络是图像分类领域的主流算法之一，但是其训练需要大量标注数据，在阿尔茨海默病医学影像等小数据集上易出现过拟合现象。数据增强算法可以用于扩充数据量，其中 CutMix 数据增强算法近来被广泛应用，但是现有方法生成的增强图像往往忽略原始图像显著区域，且增强图像的标签设计考虑的因素较为单一。针对这些问题，提出Fit CutMix数据增强算法。该算法一是利用基于显著性极值迁移的区域替换策略生成增强样本，集中源样本与目标样本中显著性高的区域；二是综合源样本与目标样本的面积和显著性信息赋予增强样本标签，为卷积神经网络提供有效的监督信息。实验结果表明，将Fit CutMix数据增强算法用于ResNet50对阿尔茨海默病进行诊断时，准确率达96.6%，比直接使用ResNet50提高了约7%，且比应用现有数据增强算法至少提高3%，可见Fit CutMix数据增强算法可以有效提高深度卷积神经网络对医学影像识别的准确率。

关键词: 数据增强, 卷积神经网络, 阿尔茨海默病, 显著性检测

Abstract:

Deep convolutional neural network is one of the mainstream algorithms in the field of image classification, but its training requires a large number of labeled data, which leads to over fitting on small datasets such as Alzheimer's medical images.Data augmentation can increase the amount of training data, and CutMix data augmentation algorithm has been widely used recently.However, the augmented images generated by the CutMix series methods often ignore the significant area of the original image, and the design of the label of the augmented image takes only single factor into consideration.In order to solve these problems, the Fit CutMix data augmentation algorithm was proposed.Firstly, the region replacement strategy based on the transfer of saliency extreme value was used to generate augmented samples, so as to concentrate the regions with high saliency value in the source samples and target samples.Secondly, the area and saliency information of the source samples and the target samples were combined to assign the augmented sample label, which provided effective supervision information for the convolutional neural network.The experimental results showed that when Fit CutMix was used in ResNet50 to diagnose Alzheimer's disease, the accuracy was 96.6%, which was about 7% higher than that of directly using ResNet50, and at least 3% higher than that of applying existing methods.Therefore, the Fit CutMix data augmentation algorithm can effectively improve the recognition accuracy of deep convolutional neural network for medical images.

Key words: data augmentation, convolutional neural network, Alzheimer's disease, saliency detection

中图分类号:

TP399

罗欣欢, 王奕璇, 李炜, 等. 基于显著性信息的Fit CutMix数据增强算法在医学影像上的应用[J]. 智能科学与技术学报, 2023, 5(1): 58-68.

Xinhuan LUO, Yixuan WANG, Wei LI, et al. Application of Fit CutMix data augmentation algorithm based on saliency information in medical images[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 58-68.

图/表 11

图1

图2

图3

表1

表2

Fit CutMix数据增强算法在AD、NC二分类任务中不同神经网络的实验结果对比"

算法	Acc	Sen	Spe	Pre	F1值	AUC
ResNet18	91.86%	93.33%	90.86%	87.51%	90.32	92.10
ResNet18+Aug	93.90%	95.00%	93.71%	91.50%	93.08	94.36
VGG11	90.51%	93.33%	88.57%	84.84%	88.88	90.95
VGG11+Aug	91.53%	95.83%	88.57%	85.19%	90.20	92.20
VGG19	84.41%	90.00%	80.57%	76.37%	82.27	85.29
VGG19+Aug	85.76%	95.00%	79.43%	76.15%	84.47	87.21
DensNet121	92.54%	91.67%	93.14%	90.26%	90.93	92.40
DensNet121+Aug	93.22%	94.17%	92.57%	90.07%	91.94	93.37
ResNet50	89.83%	91.67%	88.57%	85.56%	88.19	90.12
$R e s N e t 50 + A u g$	$96 . 61 %$	$95 . 83 %$	$97 . 14 %$	$95 . 93 %$	$95 . 87$	$96 . 49$

表2

图4

图5

图6

图7

表3

Fit CutMix算法在AD三分类任务中不同神经网络的实验结果对比"

算法	Acc	Sen	Pre-AD	Pre-MCI	Pre-NC	F1值
ResNet18	85.91%	86.94%	84.71%	83.73%	84.57%	86.58
$R e s N e t 18 + A u g$	$87 . 50 %$	$88 . 45 %$	$86 . 28 %$	$85 . 98 %$	$85 . 23 %$	$88 . 06$
VGG11	78.41%	78.98%	76.23%	74.61%	76.72%	78.51
VGG11+Aug	81.59%	82.58%	78.45%	77.98%	78.10%	82.33
VGG19	79.09%	79.98%	77.09%	76.01%	78.78%	79.43
VGG19+Aug	82.95%	83.57%	80.42%	80.12%	80.53%	83.41
DensNet121	82.50%	82.99%	79.23%	78.67%	80.71%	83.27
DensNet121+Aug	85.00%	85.98%	82.22%	82.20%	81.39%	85.59
ResNet50	80.68%	81.20%	76.80%	75.38%	76.49%	81.50
ResNet50+Aug	85.23%	85.96%	79.92%	78.95%	80.41%	86.22

表3

表4

参考文献 23

[24]	TALO M , BALOGLU U B , YILDIRIM ? ,et al. Application of deep transfer learning for automated brain abnormality classification using MR images[J]. Cognitive Systems Research, 2019,54: 176-188.
[25]	TAYLOR L , NITSCHKE G . Improving deep learning with generic data augmentation[C]// Proceedings of 2018 IEEE Symposium Series on Computational Intelligence. Piscataway:IEEE Press, 2019: 1542-1547.
[26]	ZHONG Z , ZHENG L , KANG G ,et al. Random erasing data augmentation[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2020,34(7): 13001-13008.
[27]	Asuncion A , Newman D Asuncion A,Newman D.UCL machine learning repository[Z]. 2007.
[28]	YUN S , HAN D , CHUN S ,et al. CutMix:regularization strategy to train strong classifiers with localizable features[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2020: 6022-6031.
[29]	GONG C Y , WANG D L , LI M ,et al. KeepAugment:a simple information-preserving data augmentation approach[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 1055-1064.
[30]	UDDIN A F M S , MONIRA M S , SHIN W ,et al. SaliencyMix:a saliency guided data augmentation strategy for better regularization[J]. arXiv preprint, 2020,arXiv:2006.01791.
[31]	PARK J , YANG J Y , SHIN J ,et al. Saliency grafting:innocuous attribution-guided mixup with calibrated label mixing[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022,36(7): 7957-7965.
[32]	SELVARAJU R R , COGSWELL M , DAS A ,et al. Grad-CAM:visual explanations from deep networks via gradient-based localization[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 618-626.
[33]	YAN C G , ZANG Y F . DPARSF:a MATLAB toolbox for “pipeline”data analysis of resting-state fMRI[J]. Frontiers in Systems Neuroscience, 2010,4:13.
[34]	SONG X W , DONG Z Y , LONG X Y ,et al. REST:a toolkit for resting-state functional magnetic resonance imaging data processing[J]. PLoS One, 2011,6(9).
[35]	PASZKE A , GROSS S , MASSA F ,et al. PyTorch:an imperative style,high-performance deep learning library[J]. Advances in Neural Information Processing Systems, 2019,32.
[36]	PRECHELT L . Automatic early stopping using cross validation:quantifying the criteria[J]. Neural Networks, 1998,11(4): 761-767.
[37]	DE BOER P T , KROESE D P , MANNOR S ,et al. A tutorial on the cross-entropy method[J]. Annals of Operations Research, 2005,134(1): 19-67.
[38]	张钹 . 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019,1(1): 4-6.
	ZHANG B . Artificial intelligence is entering the post deep-learning era[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 4-6.
[1]	STEVENS G A , HOGAN D R , BOERMA T . Improving reporting of health estimates[J]. Bulletin of the World Health Organization, 2016,94(7): 483.
[2]	SARRAF S , DESOUZA D , ANDERSON J A E ,et al. DeepAD:Alzheimer’s disease classification via deep convolutional neural networks using MRI and fMRI[J]. BioRxiv, 2017.
[4]	郑南宁 . 人工智能新时代[J]. 智能科学与技术学报, 2019,1(1): 1-3.
	ZHENG N N . The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(1): 1-3.
[5]	TAQI A M , AWAD A , AL-AZZO F ,et al. The impact of multi-optimizers and data augmentation on TensorFlow convolutional neural network performance[C]// Proceedings of 2018 IEEE Conference on Multimedia Information Processing and Retrieval. Piscataway:IEEE Press, 2018: 140-145.
[6]	QIAO J P , LYU Y R , CAO C F ,et al. Multivariate deep learning classification of Alzheimer’s disease based on hierarchical partner matching independent component analysis[J]. Frontiers in Aging Neuroscience, 2018,10:417.
[7]	LIN W M , TONG T , GAO Q Q ,et al. Convolutional neural networks-based MRI image analysis for the Alzheimer’s disease prediction from mild cognitive impairment[J]. Frontiers in Neuroscience, 2018,12:777.
[8]	AWATE G . Detection of Alzheimers disease from MRI using convolutional neural networks,exploring transfer learning and BellCNN[J]. arXiv preprint, 2019,arXiv:1901.10231.
[9]	MURUGAN S , VENKATESAN C , SUMITHRA M G ,et al. DEMNET:a deep learning model for early diagnosis of Alzheimer diseases and dementia from MR images[J]. IEEE Access, 2021,9: 90319-90329.
[10]	REN F J , YANG C H , QIU Q ,et al. Exploiting discriminative regions of brain slices based on 2D CNNs for Alzheimer’s disease classification[J]. IEEE Access, 2019,7: 181423-181433.
[11]	ZHANG Y T , TENG Q Z , LIU Y Y ,et al. Diagnosis of Alzheimer’s disease based on regional attention with sMRI gray matter slices[J]. Journal of Neuroscience Methods, 2022,365.
[12]	SARRAF S , TOFIGHI G . Deep learning-based pipeline to recognize Alzheimer’s disease using fMRI data[C]// Proceedings of 2016 Future Technologies Conference. Piscataway:IEEE Press, 2017: 816-820.
[13]	FAROOQ A , ANWAR S , AWAIS M ,et al. A deep CNN based multi-class classification of Alzheimer’s disease using MRI[C]// Proceedings of 2017 IEEE International Conference on Imaging Systems and Techniques. Piscataway:IEEE Press, 2018: 1-6.
[14]	KAZEMI Y , HOUGHTEN S . A deep learning pipeline to classify different stages of Alzheimer’s disease from fMRI data[C]// Proceedings of 2018 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology. Piscataway:IEEE Press, 2018: 1-8.
[15]	JANGHEL R R , RATHORE Y K . Deep convolution neural network based system for early diagnosis of Alzheimer’s disease[J]. IRBM, 2021,42(4): 258-267.
[16]	WU C L , GUO S W , HONG Y J ,et al. Discrimination and conversion prediction of mild cognitive impairment using convolutional neural networks[J]. Quantitative Imaging in Medicine and Surgery, 2018,8(10): 992-1003.
[17]	JAIN R , JAIN N , AGGARWAL A ,et al. Convolutional neural network based Alzheimer’s disease classification from magnetic resonance brain images[J]. Cognitive Systems Research, 2019,57(C): 147-159.
[18]	BILLONES C D , DEMETRIA O J L D , HOSTALLERO D E D ,et al. DemNet:a convolutional neural network for the detection of Alzheimer’s disease and mild cognitive impairment[C]// Proceedings of 2016 IEEE Region 10 Conference. Piscataway:IEEE Press, 2017: 3724-3727.
[19]	VALLIANI A , SONI A . Deep residual nets for improved Alzheimer’s diagnosis[C]// Proceedings of the 8th ACM International Conference on Bioinformatics,Computational Biology,and Health Informatics. New York:ACM Press, 2017:615.
[20]	ISLAM J , ZHANG Y Q . A novel deep learning based multi-class classification method for Alzheimer’s disease detection using brain MRI data[C]// Proceedings of International Conference on Brain Informatics. Cham:Springer, 2017: 213-222.
[21]	ISLAM J , ZHANG Y Q . Deep convolutional neural networks for automated diagnosis of Alzheimer’s disease and mild cognitive Impairment using 3D brain MRI[C]// Proceedings of International Conference on Brain Informatics. Cham:Springer, 2018: 359-369.
[22]	HON M , KHAN N M . Towards Alzheimer’s disease classification through transfer learning[C]// Proceedings of 2017 IEEE International Conference on Bioinformatics and Biomedicine. Piscataway:IEEE Press, 2017: 1166-1169.
[23]	FAROOQ A , ANWAR S , AWAIS M ,et al. Artificial intelligence based smart diagnosis of Alzheimer’s disease and mild cognitive impairment[C]// Proceedings of 2017 International Smart Cities Conference. Piscataway:IEEE Press, 2017: 1-4.

分组	AD	MCI	NC
数量/人	116	99	174
年龄/岁	74.6 ± 7.5	73.4 ± 16.0	75.5 ± 6.1

	源图像—NC样本	目标图像—AD样本	增强图像
原图
AD类别的Grad-CAM图
NC类别的Grad-CAM图

[1]	顾宙瑜, 於跃成, 者甜甜. Rapider-YOLOX：高效的轻量级目标检测网络[J]. 智能科学与技术学报, 2023, 5(1): 92-103.
[2]	邵虹, 张鸣坤, 崔文成. 基于分层卷积神经网络的皮肤镜图像分类方法[J]. 智能科学与技术学报, 2021, 3(4): 474-481.
[3]	李颖, 陈龙, 黄钊宏, 孙杨, 蔡国榕. 基于多尺度卷积神经网络特征融合的植株叶片检测技术[J]. 智能科学与技术学报, 2021, 3(3): 304-311.
[4]	张阳, 胡月, 辛东嵘. 一种考虑时空关联的深度学习短时交通流预测方法[J]. 智能科学与技术学报, 2021, 3(2): 172-178.
[5]	张国宾,王新迎. 基于混合神经网络的光伏组件输出特性数据驱动建模方法[J]. 智能科学与技术学报, 2020, 2(2): 169-178.
[6]	郝晓辰,冀亚坤,郑立召,史鑫,赵彦涛. 基于差分进化的水泥烧成系统动态优化算法[J]. 智能科学与技术学报, 2020, 2(2): 186-193.
[7]	刘文华,李浥东,王涛,邬俊,金一. 基于高维特征表示的交通场景识别[J]. 智能科学与技术学报, 2019, 1(4): 392-399.

基于显著性信息的Fit CutMix数据增强算法在医学影像上的应用

Application of Fit CutMix data augmentation algorithm based on saliency information in medical images

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 23

相关文章 7

Metrics

推荐阅读 0