神经结构搜索进展概述

doi:10.11959/j.issn.1000-0801.2019097

Abstract

Abstract:

Recently,deep learning has achieved impressive success on various computer vision tasks.The neural architecture is usually a key factor which directly determines the performance of the deep learning algorithm.The automated neural architecture search methods have attracted more and more attentions in recent years.The neural architecture search is the automated process of seeking the optimal neural architecture for specific tasks.Currently,the neural architecture search methods have shown great potential in exploring high-performance and high-efficiency neural architectures.In this paper,a survey in this research field and categorize existing methods based on their performance estimation methods,search spaces and architecture search strategies were presented.Specifically,there were four performance estimation methods for computation cost reduction,two typical neural architecture search spaces and two types of search strategies based on discrete and continuous spaces respectively.Neural architecture search methods based on continuous space are becoming the trend of researches on neural architecture search.

Key words: neural architecture search, deep learning, reinforcement learning

CLC Number:

TP183

Mingjie HE,Jie ZHANG,Shiguang SHAN. A survey of neural architecture search[J]. Telecommunications Science, 2019, 35(5): 43-50.

Figures/Tables 5

References 29

[1]	DENG J , DONG W , SOCHER R ,et al. ImageNet:a large-scale hierarchical image database[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 20-25,2009,Miami,Florida,USA. Piscataway:IEEE Press, 2009.
[2]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems (NIPS),Dec 3-8,2012,Lake Tahoe,Nevada,USA. New York:Curran Associates Inc, 2012.
[3]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2004.
[4]	SZEGEDY C , LIU W , JIA Y ,et al. Going deeper with convolutions[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 7-12,2015,Boston,MA,USA. Piscataway:IEEE Press, 2015.
[5]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 27-30,2016,Las Vegas,NV,USA. Piscataway:IEEE Press, 2016.
[6]	HE K , ZHANG X , REN S ,et al. Identity mappings in deep residual networks[C]// European Conference on Computer Vision (ECCV),Oct 11-14,2016,Amsterdam,Netherlands. Cham:Springer, 2016.
[7]	HUANG G , LIU Z , VAN DER MAATEN L ,et al. Densely connected convolutional networks[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),July 21-26,2017,Honolulu,HI,USA. Piscataway:IEEE Press, 2017.
[8]	XIE S , GIRSHICK R , DOLL R P ,et al. Aggregated residual transformations for deep neural networks[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),July 21-26,2017,Honolulu,HI,USA. Piscataway:IEEE Press, 2017.
[9]	CHEN Y , LI J , XIAO H ,et al. Dual path networks[C]// Advances in Neural Information Processing Systems (NIPS),Dec 4-9,2017,Long Beach,CA,USA. New York:Curran Associates Inc, 2017.
[10]	HU J , SHEN L , SUN G . Squeeze-and-excitation networks[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake,UT,USA. Piscataway:IEEE Press, 2018.
[11]	ELSKEN T , METZEN J H , HUTTER F . Neural architecture search:A survey[J]. arXiv:1808.05377, 2018.
[12]	LIU C , ZOPH B , NEUMANN M ,et al. Progressive neural architecture search[C]// European Conference on Computer Vision (ECCV),Sep 8-14,2018,Munich,Germany. Cham:Springer, 2018.
[13]	LUO R , TIAN F , QIN T ,et al. Neural architecture optimization[C]// Advances in Neural Information Processing Systems (NIPS),Dec 3-8,2018,Montreal,Canada. New York:Curran Associates,Inc, 2018.
[14]	PHAM H , GUAN M Y , ZOPH B ,et al. Efficient neural architecture search via parameter sharing[C]// International Conference on Machine Learning (ICML),July 10-15,2018,Stockholmsm?ssan,Stockholm,Sweden.[S.l.:sn]. 2018.
[15]	ZHONG Z , YAN J , WU W ,et al. Practical block-wise neural network architecture generation[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA. Piscataway:IEEE Press, 2018.
[16]	ZOPH B , VASUDEVAN V , SHLENS J ,et al. Learning transferable architectures for scalable image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA. Piscataway:IEEE Press, 2018.
[17]	TAN M , CHEN B , PANG R ,et al. Mnasnet:platform-aware neural architecture search for mobile[J]. arXiv:1807.11626, 2018.
[18]	REAL E , AGGARWAL A , HUANG Y ,et al. Regularized evolution for image classifier architecture search[C]// Thirty-Third AAAI Conference on Artificial Intelligence,Jan 27-Feb 1,2019,Honolulu,Hawaii,USA.[S.l.:sn]. 2019.
[19]	CAI H , YANG J , ZHANG W ,et al. Path-level network transformation for efficient architecture search[C]// International Conference on Machine Learning (ICML),July 10-15,2018,Stockholmsm?ssan,Stockholm,Sweden.[S.l.:sn]. 2018.
[20]	CHRABASZCZ P , LOSHCHILOV I , HUTTER F . A downsampled variant of imagenet as an alternative to the cifar datasets[J]. arXiv:1707.08819, 2017.
[21]	RAWAL A , MIIKKULAINEN R . From nodes to networks:evolving recurrent neural networks[J]. arXiv:1803.04439, 2018.
[22]	KRIZHEVSKY A , HINTON G . Learning multiple layers of features from tiny images[R]. 2009.
[23]	SANDLER M , HOWARD A , ZHU M ,et al. Mobilenetv2:inverted residuals and linear bottlenecks[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 18-22,2018,Salt Lake City,UT,USA. Piscataway:IEEE Press, 2018.
[24]	ZOPH B , LE Q V . Neural architecture search with reinforcement learning[C]// International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:sn]. 2017.
[25]	BAKER B , GUPTA O , NAIK N ,et al. Designing neural network architectures using reinforcement learning[C]// International Conference on Learning Representations (ICLR),April 24-26,2017,Toulon,France.[S.l.:sn]. 2017.
[26]	CAI H , CHEN T , ZHANG W ,et al. Reinforcement learning for architecture search by network transformation[J]. arXiv:1707.04873, 2017.
[27]	LIU H , SIMONYAN K , VINYALS O ,et al. Hierarchical representations for efficient architecture search[C]// International Conference on Learning Representations (ICLR),April 30-May 3,2018,Vancouver,BC,Canada.[S.l.:s. n], 2018.
[28]	REAL E , MOORE S , SELLE A ,et al. Large-scale evolution of image classifiers[C]// International Conference on Machine Learning (ICML),Aug 6-11,2017,Sydney,NSW,Australia.[S.l.:sn]. 2017.
[29]	XIE L , YUILLE A . Genetic CNN[C]// IEEE International Conference on Computer Vision (ICCV),Oct 22-29,2017,Venice,Italy. Piscataway:IEEE Press, 2017.

Metrics

Recommended 0

No Suggested Reading articles found!

方法名称	优化目标	搜索空间	搜索策略	参数数量/MB	CIFAR-10 Top1 Error	GPU数量	GPU运行天数
NASNet^[16]	精度	基于cell结构	离散，强化学习	27.6	2.97%	500	4
PNASNet^[12]	精度	基于cell结构	离散，强化学习	3.2	3.63%^[14]	100^[14]	1.5^[14]
MNASNet^[17]*	精度，速度	基于chain结构	离散，强化学习	4.4	-	-	-
ENAS^[14]	精度	基于cell结构	离散，强化学习	4.6	3.54%	1	0.45
BlockQNN^[15]	精度	基于cell结构	离散，强化学习	39.8	3.54%	32	3
AmoebaNet^[18]	精度	基于cell结构	离散，进化算法	3.2	3.34%	450	7
NAONet^[13]	精度	基于cell结构	连续优化	28.6	2.98%	200	1

A survey of neural architecture search

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 29

Related Articles 15

Metrics

Recommended 0

[1]	Min LU, Juan HU, Xianchao ZHANG, Weijian DING, Guangxue YUE. Personalized recommendation model based on users multi-features fusion [J]. Telecommunications Science, 2023, 39(5): 101-115.
[2]	Xuerong WANG, Zhengzhi TANG, Yinchuan LI, Meiyu QI, Jianbo ZHU, Liang ZHANG. Delay-sensitive traffic intellisense scheduling based on optimal decision tree [J]. Telecommunications Science, 2023, 39(4): 120-132.
[3]	Xiwen LIAO, Supeng LENG, Yujun MING, Tianyang LI. Digital twin based intelligent urban traffic forecasting and guidance strategy [J]. Telecommunications Science, 2023, 39(3): 70-79.
[4]	Yu KANG, Yaqiong LIU, Tongyu ZHAO, Guochu SHOU. A survey on AI algorithms applied in communication and computation in Internet of vehicles [J]. Telecommunications Science, 2023, 39(1): 1-19.
[5]	Bin ZHUGE, Zhenghu YIN, Wenxue SI, Lei YAN, Ligang DONG, Xian JIANG. Student knowledge tracking based multi-indicator exercise recommendation algorithm [J]. Telecommunications Science, 2022, 38(9): 129-143.
[6]	Jie ZHOU, Bernardo Esono Esono Mikue, Xueying WANG, Huiting ZHOU, Hong LUO. PAPR optimization based on SLM and PTS algorithms in NC-OFDM systems [J]. Telecommunications Science, 2022, 38(7): 63-74.
[7]	Xin LI, Jun SUN. Value-difference learning based mMTC devices access algorithm in multi-cell network [J]. Telecommunications Science, 2022, 38(6): 82-90.
[8]	Panpan LI, Zhengxia XIE, Guangxue YUE, Xin LIU. Research progress and trends of deep learning based wireless communication receiving method [J]. Telecommunications Science, 2022, 38(2): 1-17.
[9]	Qing SHEN, Wenbin GUO, Jungang LOU, Qiangguo YU. Personalized recommendation model with multi-level latent features [J]. Telecommunications Science, 2022, 38(2): 71-83.
[10]	Nan JIN, Ruiqin WANG, Yuecong LU. Ebbinghaus forgetting curve and attention mechanism based recommendation algorithm [J]. Telecommunications Science, 2022, 38(10): 89-97.
[11]	Zhihong CHEN, Mingxiao WANG. Application of computer vision in intelligent security [J]. Telecommunications Science, 2021, 37(8): 142-147.
[12]	Jianwu ZHANG, Luxin WANG, Lingfen SUN, Qianye ZHANG, Hangguan SHAN. An survey on application of artificial intelligence in 5G system [J]. Telecommunications Science, 2021, 37(5): 14-31.
[13]	Boheng TANG, Xingang CHAI. Cloud-edge collaboration based computer vision inference mechanism [J]. Telecommunications Science, 2021, 37(5): 72-81.
[14]	Shujun SUN, Shengliang PENG, Yudong YAO, Xi YANG. A survey of deep learning based modulation recognition [J]. Telecommunications Science, 2021, 37(5): 82-90.
[15]	Shuang PENG, Xiaodong WANG, Zongju PENG, Fen CHEN. Fast QTMT partition decision based on deep learning [J]. Telecommunications Science, 2021, 37(4): 73-81.