深度学习编程框架

doi:10.11959/j.issn.2096-0271.2018040

大数据 ›› 2018, Vol. 4 ›› Issue (4): 56-63.doi: 10.11959/j.issn.2096-0271.2018040

• 专题：大数据驱动的智能计算体系架构 • 上一篇下一篇

深度学习编程框架

王秉睿^1,²,兰慧盈^2,³,陈云霁²

¹ 中国科学技术大学，安徽合肥 230026
² 中国科学院计算技术研究所，北京 100190
³ 中国科学院大学，北京 100049

出版日期:2018-07-15 发布日期:2018-08-08
作者简介:王秉睿（1994-），男，中国科学技术大学硕士生，主要研究方向为计算机体系结构、机器学习编程方法、人工智能。|兰慧盈（1990-），女，中国科学院计算技术研究所博士生，主要研究方向为计算机体系结构、领域专用编程语言和编译器、人工智能。|陈云霁（1983-），男，博士，中国科学院计算技术研究所研究员（正教授）、博士生导师，主要研究方向为计算机体系结构、人工智能。
基金资助:
国家重点研发计划基金资助项目(2017YFB1003101);国家自然科学基金资助项目(61472396);国家自然科学基金资助项目(61432016);国家自然科学基金资助项目(61473275);国家自然科学基金资助项目(61522211);国家自然科学基金资助项目(61532016);国家自然科学基金资助项目(61521092);国家自然科学基金资助项目(61502446);国家自然科学基金资助项目(61672491);国家自然科学基金资助项目(61602441);国家自然科学基金资助项目(61602446);国家自然科学基金资助项目(61732002);国家自然科学基金资助项目(61702478);北京市科技计划基金资助项目(Z151100000915072);中国科学院STS计划；国家重点基础研究发展计划（“973”计划）基金资助项目(2015CB358800)

Programming frameworks for deep learning algorithms

Bingrui WANG^1,²,Huiying LAN^2,³,Yunji CHEN²

¹ University of Science and Technology of China,Hefei 230026,China
² Institute of Computing Technology Chinese Academy of Sciences,Beijing 100190,China
³ University of Chinese Academy of Sciences,Beijing 100049,China

Online:2018-07-15 Published:2018-08-08
Supported by:
National Key Research and Development Program of China(2017YFB1003101);The National Netural Science Foundation of China(61472396);The National Netural Science Foundation of China(61432016);The National Netural Science Foundation of China(61473275);The National Netural Science Foundation of China(61522211);The National Netural Science Foundation of China(61532016);The National Netural Science Foundation of China(61521092);The National Netural Science Foundation of China(61502446);The National Netural Science Foundation of China(61672491);The National Netural Science Foundation of China(61602441);The National Netural Science Foundation of China(61602446);The National Netural Science Foundation of China(61732002);The National Netural Science Foundation of China(61702478);Beijing Science and Technology Projects(Z151100000915072);STS Project of Chinese Academy of Sciences,The National Key Basic Research Program of China(973 Program)(2015CB358800)

摘要/Abstract

摘要：

近年来，深度学习算法日益流行，在各种领域的应用都取得了出色的效果，受到工业界和学术界的广泛关注。越来越多的研究者开始利用深度学习算法解决实际问题（如图像分类、图像识别、语音识别、自然语言处理等）。人们提出了各种各样的深度学习编程框架，便于研究者们开发新的深度学习算法。这些深度学习库的设计原则、抽象层次各有不同。对常见的深度学习编程框架进行了分类介绍，针对深度学习编程框架设计中的关键问题进行了分析，并且对未来深度学习编程框架的发展方向做了展望，为以后深度学习编程框架的设计提供了设计思路和方向。

关键词: 机器学习, 编程框架, 大数据

Abstract:

In recent years,deep learning algorithms became increasingly pervasive.It has drew extensive attentions from both researchers and industries,as it achieves very promising results on many applications of various fields.More and more researchers began to use deep learning algorithms to solve practical problems (e.g.,image classification,image recognition,speech recognition,and natural language processing).Many deep learning frameworks and libraries were proposed so that researchers can develop new deep learning algorithms in a more convenient fashion.These frameworks and libraries were different in many aspects (e.g.,design principles and abstraction).Firstly,several pervasive deep learning frameworks were introduced,and then the critical issue of designing such frameworks was analyzed.At last,the future challenges of designing deep learning frameworks were discussed.The study provides ideas and directions for future design.

Key words: machine learning, programming framework, big data

中图分类号:

F830

王秉睿, 兰慧盈, 陈云霁. 深度学习编程框架[J]. 大数据, 2018, 4(4): 56-63.

Bingrui WANG, Huiying LAN, Yunji CHEN. Programming frameworks for deep learning algorithms[J]. Big Data Research, 2018, 4(4): 56-63.

图/表 2

参考文献 22

[1]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),June 26-July 1,2016,Las Vegas,USA. Piscataway:IEEE Press, 2016: 770-778.
[2]	ABADI M , BARHAM P , CHEN J ,et al. TensorFlow:a system for large-scale machine learning[C]// The 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16),November 2-4,2016,Savannah,USA. Berkeley:USENIX Association, 2016: 265-284.
[3]	JOUPPI N P , YOUNG C , PATIL N ,et al. In-datacenter performance analysis of a tensor processing unit[C]// The 44th Annual International Symposium on Computer Architecture,June 24-28,2017,Toronto,Canada. New York:ACM Press, 2017: 1-12.
[4]	ZHANG M , ZHOU Z . ML-KNN :a lazy learning approach to multi-label learning[J]. Pattern Recognition, 2007,40(7): 2038-2048.
[5]	YANG S L , LI Y S , HU X X ,et al. Optimization study on k value of kmeans algorithm[J]. Systems EngineeringTheory ＆ Practice, 2006,26(2): 97-101.
[6]	MADIGAN D , YORK J . Bayesian graphical models for discrete data[J]. International Statistical Review, 1995,63(2): 215-232.
[7]	OSUNA E , FREUND R , GIROSI F . Training svm:an application to face detection[R]. 1997.
[8]	LECUN Y , KAVUKCUOGLU K , FARABET C . Convolutional networks and applications in vision[C]// International Symposium on Circuits and Systems,May 30-June 2,2010,Paris,France. Piscataway:IEEE Press, 2010: 253-256.
[9]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[C]// The 25th International Conference on Neural Information Processing Systems,December 3-6,2012,Lake Tahoe,Nevada.New York:Curran Associates Inc. , 2012: 1-9.
[10]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. Computer Science, 2014:arXiv:1409.1556.
[11]	CHEN T , LI M , LI Y ,et al. MXNet :a flexible and efficient machine learning library for heterogeneous distributed systems[J]. Statistics, 2015:arXiv:1512.01274.
[12]	TEAM T D , ALRFOU R , ALAIN G ,et al. Theano:a Python framework for fast computation of mathematical expressions[J]. Computer Science, 2016:arXiv:1605.02688.
[13]	COLLOBERT R , KAVUKCUOGLU K , FARABET C . Torch7:a matlab-like environment for machine learning[C]// The 25th Annual Conference on Neural Information Processing Systems,December 12-14,2011,Granada,Spain.[S.l.:s.n]. 2011: 1-6.
[14]	JIA Y , SHELHAMER E , DONAHUE J ,et al. Caffe:convolutional architecture for fast feature embedding[J]. Computer Science, 2014:arXiv:1408.5093.
[15]	CHEN T , DU Z , SUN N ,et al. DianNao:a small-footprint high-throughput accelerator for ubiquitous machinelearning[C]// The 19th International Conference on Architectural Support for Programming Languages and Operating Systems,March 1-5,2014,Salt Lake City,USA. New York:ACM Press, 2014: 269-284.
[16]	CHEN Y , LUO T , LIU S ,et al. DaDianNao:a machine-learning supercomputer[C]// The 47th Annual IEEE/ACM International Symposium on Microarchitecture,December 13-17,2014,Cambridge,UK. Washington DC:IEEE Computer Society, 2014: 609-622.
[17]	DU Z , FASTHUBER R , CHEN T ,et al. ShiDianNao[C]// The 42nd Annual International Symposium on Computer Architecture,June 13-17,2015,Portland,USA.[S.l.:s.n]. 2015: 92-104.
[18]	LIU D , CHEN T , LIU S ,et al. PuDianNao:a polyvalent machine learning accelerator[C]// The 20th International Conference on Architectural Support for Programming Languages and Operating Systems,March 14-18,2015,Istanbul,Turkey. New York:ACM Press, 2015: 369-381.
[19]	ZHANG S , DU Z , ZHANG L ,et al. Cambricon-X:an accelerator for sparse neural networks[C]// The 49th Annual IEEE/ACM International Symposium on Microarchitecture,October 15-19,2016,Taipei,China. Piscataway:IEEE Press, 2016: 1-12.
[20]	LIU S , DU Z , TAO J ,et al. Cambricon:an instruction set architecture for neural networks[C]// The 43rd International Symposium on Computer Architecture,June 18-22,2016,Seoul,Korea. New York:ACM Press, 2016: 393-405.
[21]	WEI R , SCHWARTZ L , ADVE V . DLVM:a modern compiler infrastructure for deep learning systems[J]. Computer Science, 2017:arXiv:1711.03016.
[22]	CHEN T , MOREAU T , JIANG Z ,et al. TVM:end-to-end optimization stack for deep learning[J]. Computer Science, 2018:arXiv:1802.04799v1.

类别	算法	应用
经典机器学习算法	k-NN^[4]	聚类（无监督学习）
	k-means^[5]、决策树、朴素贝叶斯^[6]、逻辑回归、线性回归、支持向量机^[7]	分类、回归（有监督学习）
	主成分分析（principle component analysis，PCA）	数据降维
深度学习算法	卷积神经网络（convolutional neural network，CNN）	图像识别、图像分类、语音识别
	递归神经网络（recurrent neural network，RNN）/ 长短期记忆（long short-term memory，LSTM）网络	自然语言处理

深度学习编程框架

Programming frameworks for deep learning algorithms

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 2

参考文献 22

相关文章 15

Metrics

推荐阅读 0

[1]	钱海红, 王茂异, 熊贇. 高等教育数字化转型的现状与发展研究[J]. 大数据, 2023, 9(3): 56-70.
[2]	梅宏, 杜小勇, 金海, 程学旗, 柴云鹏, 石宣化, 靳小龙, 王亚沙, 刘驰. 大数据技术前瞻[J]. 大数据, 2023, 9(1): 1-20.
[3]	沈阳, 余梦珑. 元宇宙与大数据：时空智能中的数据洞察与价值连接[J]. 大数据, 2023, 9(1): 103-110.
[4]	陈静. 人文大数据及其在数字人文领域中的应用[J]. 大数据, 2022, 8(6): 3-14.
[5]	罗煜楚, 吴昊, 郭宇涵, 谭绍聪, 刘灿, 蒋瑞珂, 袁晓如. 数字人文中的可视化[J]. 大数据, 2022, 8(6): 74-93.
[6]	张伶俐, 褚琦凯, 王桂娟, 张巍瀚, 蒲慧, 宋振金, 吴亚东. 文本情感可视分析技术及其在人文领域的应用[J]. 大数据, 2022, 8(6): 56-73.
[7]	郑童哲恒, 李斌, 冯敏萱, 常博林, 王东波. 历史典籍的结构化探索——《史记·列传》数字人文知识库的构建与可视化研究[J]. 大数据, 2022, 8(6): 40-55.
[8]	吴建汉, 司世景, 王健宗, 肖京. 联邦学习攻击与防御综述[J]. 大数据, 2022, 8(5): 12-32.
[9]	李汶龙, 袁媛, 安筱鹏. 刍议大数据治理的三大基础思维[J]. 大数据, 2022, 8(4): 34-45.
[10]	汤奇峰, 邵志清, 叶雅珍. 数据交易中的权利确认和授予体系[J]. 大数据, 2022, 8(3): 40-53.
[11]	王陈慧子, 蔡玮. 元宇宙数字经济：现状、特征与发展建议[J]. 大数据, 2022, 8(3): 140-150.
[12]	杨玫, 李玮, 乔思渊, 刘巍. 中国大数据产业产值测算方法研究[J]. 大数据, 2022, 8(3): 151-160.
[13]	李德仁, 张过, 蒋永华, 沈欣, 刘伟玲. 论大数据视角下的地球空间信息学的机遇与挑战[J]. 大数据, 2022, 8(2): 3-14.
[14]	仇晓兰, 胡玉新, 上官松涛, 付琨. 遥感卫星大数据高精度一体化处理技术[J]. 大数据, 2022, 8(2): 15-27.
[15]	刘伟权, 王程, 臧彧, 胡倩, 于尚书, 赖柏锜. 基于遥感大数据的信息提取技术综述[J]. 大数据, 2022, 8(2): 28-57.