基于IR-VGG的多分类皮肤病实时诊断

doi:10.11959/j.issn.2096-3750.2021.00217

物联网学报 ›› 2021, Vol. 5 ›› Issue (3): 115-125.doi: 10.11959/j.issn.2096-3750.2021.00217

基于IR-VGG的多分类皮肤病实时诊断

谈玲¹, 荣杉山¹, 夏景明², SajibSarker², 马雯杰¹

¹ 南京信息工程大学计算机与软件学院，江苏南京 210044
² 南京信息工程大学人工智能学院，江苏南京 210044
³ 南京信息工程大学雷丁学院，江苏南京 210044

修回日期:2021-01-13 出版日期:2021-09-30 发布日期:2021-09-01
作者简介:谈玲（1979- ），女，南京信息工程大学计算机与软件学院助理教授，主要研究方向为机器学习和深度学习在图像处理中的应用
荣杉山（1997- ），男，南京信息工程大学计算机与软件学院硕士生，主要研究方向为深度学习在医疗图像中的应用
夏景明（1980- ），男，南京信息工程大学人工智能学院助理教授，主要研究方向为机器学习和深度学习在气象学中的应用
Sajib Sarker（1994- ），男，南京信息工程大学雷丁学院硕士生，主要研究方向为机器学习、深度学习、人工智能、计算机视觉、模式识别和图像处理
马雯杰（1997- ），女，南京信息工程大学计算机与软件学院硕士生，主要研究方向为计算机视觉、医学图像处理和深度学习
基金资助:
国家重点研发计划(2018YFC1506502)

Real-time diagnosis of multi-category skin diseases based on IR-VGG

Ling TAN¹, Shanshan RONG¹, Jingming XIA², Sarker SAJIB², Wenjie MA¹

¹ School of Computer and Software, Nanjing University of Information Science ＆Technology, Nanjing 210044, China
² School of Artificial Intelligence, Nanjing University of Information Science ＆Technology, Nanjing 210044, China
³ Reading Academy of Nanjing University of Information Science ＆Technology, Nanjing 210044, China

Revised:2021-01-13 Online:2021-09-30 Published:2021-09-01
Supported by:
The National Key Research and Development Program of China(2018YFC1506502)

摘要/Abstract

摘要：

恶性的皮肤病变在早期阶段的治愈率极高，基于深度学习的皮肤病诊断研究近年来受到持续关注，其诊断准确率较高，然而计算资源消耗大，且依赖于医院大型计算设备。为在物联网移动设备上实现快速准确皮肤病诊断，提出一种基于IR-VGG（inverted residual visual geometry group）的多分类皮肤病实时诊断系统，使用轮廓检测算法分割出皮肤病图像病灶区域，并用反转残差块替换 VGG16 第一层卷积块以降低网络参数权重和内存开销；将原图像和分割后的病灶图像输入IR-VGG网络，通过全局和局部特征提取后，输出皮肤病诊断结果。实验结果表明，IR-VGG网络结构在SkinData-1和SkinData-2皮肤病数据集上的准确率分别可达到94.71%和85.28%，并且可以有效降低复杂度，使诊断系统较容易在物联网移动设备上进行皮肤病实时诊断。

关键词: 皮肤病, 边缘检测分割, 反转残差块, 深度学习, 物联网移动设备

Abstract:

Malignant skin lesions have a very high cure rate in the early stage.In recent years, dermatological diagnosis research based on deep learning has been continuously promoted, with high diagnostic accuracy.However, computational resource consumption is huge and it relies on large computing equipment in hospitals.In order to realize rapid and accurate diagnosis of skin diseases on Internet of things (IoT) mobile devices, a real-time diagnosis system of multiple categories of skin diseases based on inverted residual visual geometry group (IR-VGG) was proposed.The contour detection algorithm was used to segment the lesion area of skin image.The convolutional block of the first layer of VGG16 was replaced with reverse residual block to reduce the network parameter weight and memory overhead.The original image and the segmented lesion image was inputed into IR-VGG network, and the dermatological diagnosis results after global and local feature extraction were outputed.The experimental results show that the IR-VGG network structure can achieve 94.71% and 85.28% accuracy in Skindata-1 and Skindata-2 skin diseases data sets respectively, and can effectively reduce complexity, making it easier for the diagnostic system to make real-time skin diseases diagnosis on IoT mobile devices.

Key words: skin lesions, edge detection segmentation, inverted residual, deep learning, Internet of things mobile devices

中图分类号:

TP391.41

谈玲, 荣杉山, 夏景明, SajibSarker, 马雯杰. 基于IR-VGG的多分类皮肤病实时诊断[J]. 物联网学报, 2021, 5(3): 115-125.

Ling TAN, Shanshan RONG, Jingming XIA, Sarker SAJIB, Wenjie MA. Real-time diagnosis of multi-category skin diseases based on IR-VGG[J]. Chinese Journal on Internet of Things, 2021, 5(3): 115-125.

图/表 12

图1

图2

表1

图3

图4

图5

表2

IR-VGG在不同数据集的分类性能评估"

数据集名称	实验名称	准确率	大小	耗时
	实验1	$94 . 71 %$
SkinData-1	实验2	88.97%	5.81 GB	16 591 s
	实验3	92.46%
	实验4	85.02%
	实验1	$85 . 28 %$
SkinData-2	实验2	80.12%	834 MB	4 692 s
	实验3	83.23%
	实验4	79.67%

表2

表3

不同学习率的性能指标"

学习率	准确率	损失率	精确率	召回率	F1分值
0.001	86.77%	0.266 5	89.44%	83.79%	86.52%
0.005	90.13%	0.196 1	90.35%	90.86%	90.61%
0.01	90.75%	0.226 2	87.81%	90.98%	89.36%
$0 . 05$	$94 . 71 %$	$0 . 1237$	$90 . 81 %$	$94 . 70 %$	$92 . 71 %$
0.1	89.76%	0.223 3	85.03%	86.47%	85.74

表3

图6

图7

表4

不同网络结构分类性能结果"

网络	改进方法	准确率	参数量	损失率
AlexNet	NoneSeg+IR	86.68% 90.43%	29.5 MB4.65 MB	0.273 20.208 9
GoogLeNet	NoneSeg+IR	86.01% 90.10%	6.76 MB6.32 MB	0.140 40.193 8
ResNet34	NoneSeg+IR	84.64% 77.88%	22.8 MB22.7 MB	0.308 70.549 9
ResNet50	NoneSeg+IR	85.84% 63.39%	45.75 MB45.8 MB	0.255 14.744 6
VGG16	None $S e g + I R$	85.02% $94 . 71 %$	138 MB $15 . 18 M B$	0.361 4 $0 . 1237$

表4

表5

IR-VGG网络与其他相关算法模型对比"

已有研究工作	分类模型	准确率	参数量	问题类型
文献[3]所提方案	BN-CNN	89.3%	/	二分类问题
文献[12]所提方案	MSM-CNN	96.3%	很大	多分类问题
文献[15]所提方案	Resnet50	86.7%	46 MB	二分类问题
文献[13]所提方案	Resnet152	86.28%	117 MB	二分类问题
文献[14]所提方案	VGG16（迁移）	91.07%	138 MB	多分类问题
本文模型	$I R - V G G$	$94 . 71 %$	$15 M B$	多分类问题

表5

参考文献 40

[1]	ATLANTA. American cancer society[N]. Cancer Facts ＆ Figures, 2018.
[2]	斯璐, 郭军 . 新版中国黑素瘤诊治指南解读[J]. 临床肿瘤学杂志, 2012,17(2): 172-173.
	SI L , GUO J . Interpretation of clinical practice guidelines for management of melanoma in China (new version of 2011)[J]. Chinese Clinical Oncology, 2012,17(2): 172-173.
[3]	KASMI R , MOKRANI K . Classification of malignant melanoma and benign skin lesions:implementation of automatic ABCD rule[J]. IET Image Processing, 2016,10(6): 448-455.
[4]	SOYER H P , ARGENZIANO G , ZALAUDEK I ,et al. Three-point checklist of dermoscopy[J]. Dermatology, 2004,208(1): 27-31.
[5]	ARGENZIANO G , CATRICALà C , ARDIGO M ,et al. Seven-point checklist of dermoscopy revisited[J]. British Journal of Dermatology, 2011,164(4): 785-790.
[6]	MENZIES S W . A method for the diagnosis of primary cutaneous melanoma using surface microscopy[J]. Dermatologic Clinics, 2001,19(2): 299-305.
[7]	HENNING J S , DUSZA S W , WANG S Q ,et al. The CASH (color,architecture,symmetry,and homogeneity) algorithm for dermoscopy[J]. Journal of the American Academy of Dermatology, 2007,56(1): 45-52.
[8]	XIE F Y , FAN H D , LI Y ,et al. Melanoma classification on dermoscopy images using a neural network ensemble model[J]. IEEE Transactions on Medical Imaging, 2017,36(3): 849-858.
[9]	ALI A R A , DESERNO T M . A systematic review of automated melanoma detection in dermatoscopic images and its ground truth data[C]// Proceedings of SPIE Medical Imaging:Image Perception,Observer Performance,and Technology Assessment. 2012.
[10]	KOROTKOV K , GARCIA R . Computerized analysis of pigmented skin lesions:a review[J]. Artificial Intelligence in Medicine, 2012,56(2): 69-90.
[11]	JAMIL U , KHALID S . Comparative study of classification techniques used in skin lesion detection systems[C]// Proceedings of 17th IEEE International Multi Topic Conference 2014. Piscataway:IEEE Press, 2014: 266-271.
[12]	BI L , KIM J , AHN E ,et al. Automatic melanoma detection via multi-scale lesion-biased representation and joint reverse classification[C]// Proceedings of 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). Piscataway:IEEE Press, 2016: 1055-1058.
[13]	HUANG A , KWAN S Y , CHANG W Y ,et al. A robust hair segmentation and removal approach for clinical images of skin lesions[C]// Proceedings of 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Piscataway:IEEE Press, 2013: 3315-3318.
[14]	SáEZ A , SERRANO C , ACHA B . Model-based classification methods of global patterns in dermoscopic images[J]. IEEE Transactions on Medical Imaging, 2014,33(5): 1137-1147.
[15]	IYATOMI H , CELEBI M E , SCHAEFER G ,et al. Automated color calibration method for dermoscopy images[J]. Computerized Medical Imaging and Graphics, 2011,35(2): 89-98.
[16]	SCHAEFER G , RAJAB M I , CELEBI M E ,et al. Colour and contrast enhancement for improved skin lesion segmentation[J]. Computerized Medical Imaging and Graphics, 2011,35(2): 99-104.
[17]	CELEBI M E , KINGRAVI H A , UDDIN B ,et al. A methodological approach to the classification of dermoscopy images[J]. Computerized Medical Imaging and Graphics, 2007,31(6): 362-373.
[18]	GANSTER H , PINZ P , ROHRER R ,et al. Automated melanoma recognition[J]. IEEE Transactions on Medical Imaging, 2001,20(3): 233-239.
[19]	XIE F Y , FAN H D , LI Y ,et al. Melanoma classification on dermoscopy images using a neural network ensemble model[J]. IEEE Transactions on Medical Imaging, 2017,36(3): 849-858.
[20]	CAPDEHOURAT G , COREZ A , BAZZANO A ,et al. Toward a combined tool to assist dermatologists in melanoma detection from dermoscopic images of pigmented skin lesions[J]. Pattern Recognition Letters, 2011,32(16): 2187-2196.
[21]	LAZEBNIK S , SCHMID C , PONCE J . Beyond bags of features:spatial pyramid matching for recognizing natural scene categories[C]// Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). Piscataway:IEEE Press, 2006: 2169-2178.
[22]	CHEN S J , DING G G , LI C X ,et al. Image representation optimization based on locally aggregated descriptors[C]// Proceedings of Advances in Knowledge Discovery and Data Mining. 2016.
[23]	SáNCHEZ J , PERRONNIN F , MENSINK T ,et al. Image classification with the fisher vector:theory and practice[J]. International Journal of Computer Vision, 2013,105(3): 222-245.
[24]	CHATFIELD K , LEMPITSKY V , VEDALDI A ,et al. The devil is in the details:an evaluation of recent feature encoding methods[C]// Proceedings of the British Machine Vision Conference(2011). 2011.
[25]	SONG Y Y , ZHANG L , CHEN S P ,et al. Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning[J]. IEEE Transactions on Biomedical Engineering, 2015,62(10): 2421-2433.
[26]	HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016: 770-778.
[27]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[28]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[29]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553): 436-444.
[30]	JAYALAKSHMI G S , KUMAR V S . Performance analysis of convolutional neural network (CNN) based cancerous skin lesion detection system[C]// Proceedings of 2019 International Conference on Computational Intelligence in Data Science (ICCIDS). Piscataway:IEEE Press, 2019: 1-6.
[31]	胡海根, 孔祥勇, 周乾伟 ,等. 基于深层卷积残差网络集成的黑色素瘤分类方法[J]. 计算机科学, 2019,46(5): 247-253.
	HU H G , KONG X Y , ZHOU Q W ,et al. Melanoma classification method by integrating deep convolutional residual network[J]. Computer Science, 2019,46(5): 247-253.
[32]	YU L Q , CHEN H , DOU Q ,et al. Automated melanoma recognition in dermoscopy images via very deep residual networks[C]// Proceedings of IEEE Transactions on Medical Imaging. Piscataway:IEEE Press, 2019: 994-1004.
[33]	MAHBOD A , SCHAEFER G , WANG C L ,et al. Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification[J]. Computer Methods and Programs in Biomedicine, 2020,193: 105475.
[34]	李航, 余镇, 倪东 ,等. 基于深度残差网络的皮肤镜图像黑色素瘤的识别[J]. 中国生物医学工程学报, 2018,37(3): 274-282.
	LI H , YU Z , NI D ,et al. Melanoma recognition in dermoscopy images via deep residual network[J]. Chinese Journal of Biomedical Engineering, 2018,37(3): 274-282.
[35]	GUHA S R , M RAFIZUL HAQUE S . Convolutional neural network based skin lesion analysis for classifying melanoma[C]// Proceedings of 2019 International Conference on Sustainable Technologies for Industry4.0 (STI). Piscataway:IEEE Press, 2019: 1-5.
[36]	CHEN M , ZHOU P , WU D ,et al. AI-Skin:skin disease recognition based on self-learning and wide data collection through a closed-loop framework[J]. Information Fusion, 2020,54: 1-9.
[37]	ZAFAR K , GILANI S O , WARIS A ,et al. Skin lesion segmentation from dermoscopic images using convolutional neural network[J]. Sensors, 2020,20(6): 1601.
[38]	MARQUES O . Gray-level transformations[M]. New Jersey: John Wiley ＆ Sons, 2011.
[39]	CHENG S H , SHANG G C , ZHANG L . Handwritten digit recognition based on improved VGG16 network[C]// Proceedings of SPIE 11069,Tenth International Conference on Graphics and Image Processing (ICGIP 2018). 2019.
[40]	CODELLA N , ROTEMBERG V , TSCHANDL P ,et al. Skin lesion analysis toward melanoma detection 2018:A challenge hosted by the international skin imaging collaboration (ISIC)[J]. arXiv preprint arXiv:1902.03368, 2019.

皮肤病种类	BCC	BKL	MEL	NV
实例
训练集样本数量	2 127	2 100	2 456	2 696
测试集样本数量	531	524	615	768

基于IR-VGG的多分类皮肤病实时诊断

Real-time diagnosis of multi-category skin diseases based on IR-VGG

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 40

相关文章 10

Metrics

推荐阅读 0

[1]	张志飞, 刘峰, 葛祎阳, 李烁, 张煜, 熊轲. 一种基于深度可分离卷积和注意力机制的入侵检测方法[J]. 物联网学报, 2023, 7(1): 49-59.
[2]	蒋锐, 孙刘婷, 王小明, 李大鹏, 徐友云. 基于AE和Transformer的运动想象脑电信号分类研究[J]. 物联网学报, 2023, 7(1): 118-128.
[3]	张彪, 汪西明, 徐逸凡, 李文, 韩昊, 刘松仪, 陈学强. 基于多智能体深度强化学习的多域协同抗干扰方法研究[J]. 物联网学报, 2022, 6(4): 104-116.
[4]	李贤, 毕宿志, 曾泓儒, 林彬, 林晓辉. 基于智能化用户协作的边缘计算任务卸载与资源分配优化[J]. 物联网学报, 2022, 6(4): 41-52.
[5]	罗丹, 徐茹枝, 关志涛. 物联网环境中基于深度学习的差分隐私预算优化方法[J]. 物联网学报, 2022, 6(2): 65-76.
[6]	徐宣哲, 宁珂, 郑学敏, 赵明心, 徐萌萌, 吴南健, 刘力源. 基于硬件仿真系统的边缘计算人工智能视觉芯片设计验证[J]. 物联网学报, 2022, 6(1): 20-28.
[7]	李国权, 徐永海, 林金朝, 黄正文. 基于Res-DNN的端到端MIMO系统信号检测算法[J]. 物联网学报, 2022, 6(1): 65-72.
[8]	林椿珉, 曾烈康, 陈旭. 边缘智能驱动的高能效无人机自主导航算法研究[J]. 物联网学报, 2021, 5(2): 87-96.
[9]	陈慕涵,郭佳佳,李潇,金石. 基于深度学习的大规模MIMO信道状态信息反馈[J]. 物联网学报, 2020, 4(1): 33-44.
[10]	廖勇,姚海梅,花远肖. 一种基于深度学习的物联网信道状态信息获取算法[J]. 物联网学报, 2019, 3(1): 8-13.