基于云边协同的计算机视觉推理机制

doi:10.11959/j.issn.1000-0801.2021107

摘要/Abstract

摘要：

深度学习和云计算的普及推动了计算机视觉在各行业中的广泛应用。但集中化的云端推理服务存在带宽资源消耗大、图像数据隐私泄露、时效性难以满足等问题，难以充分满足计算机视觉在行业应用上的多样化应用需求。而通信网络的双吉比特升级将促进视觉算法云边算法深层次协同。对基于云边协同的计算机视觉推理机制开展研究。首先对近年主流的云侧和边缘侧计算机视觉推理模型的优劣势进行了分析和阐述，然后在此基础上对云边协同计算机视觉推理模型框架、部署机制等开展研究，详细讨论模型分布式推理模型分割策略，云边协同网络部署优化策略。最后通过数据协同、网络分区协同、业务功能协同 3 方面对云边协同深度推理未来的发展挑战进行了展望。

关键词: 计算机视觉, 深度学习, 云边协同

Abstract:

The popularity of deep learning and cloud computing has promoted the widespread application of computer vision in various industries.However, centralized cloud inference services have problems such as high bandwidth resource consumption, image data privacy leakage, and high latency.It is hard that satisfy demand which requires diversified computer vision application.The dual gigabit upgrade of the communication network will promote depth collaboration of computer vision cloud-edge algorithms.Aiming to study the computer vision inference mechanism based on cloud-edge collaboration.Firstly, the advantages and disadvantages of the mainstream cloud and edge computer vision inference models in recent years were analyzed and explained, and on this basis, research on the cloud-edge collaborative computer vision inference model framework and deployment mechanism was carried out, model distributed reasoning model segmentation strategy, cloud-side collaborative network deployment optimization strategy was discussed in detail.In the end, the challenge and prospect of deep learning cloud-edge collaboration inference in future was discussed through data collaboration, network partition collaboration, and business function collaboration .

Key words: computer vision, deep learning,, cloud-edge collaboration

中图分类号:

TP393

唐博恒, 柴鑫刚. 基于云边协同的计算机视觉推理机制[J]. 电信科学, 2021, 37(5): 72-81.

Boheng TANG, Xingang CHAI. Cloud-edge collaboration based computer vision inference mechanism[J]. Telecommunications Science, 2021, 37(5): 72-81.

图/表 3

参考文献 46

[1]	VéSTIAS M P , DUARTE R P , DE SOUSA J T ,et al. Moving deep learning to the edge[J]. Algorithms, 2020,13(5): 125.
[2]	LE CUN Y , JACKEL L D , BOSER B ,et al. Handwritten digit recognition:applications of neural network chips and automatic learning[J]. IEEE Communications Magazine, 1989,27(11): 41-46.
[3]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012(25): 1097-1105.
[4]	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[5]	SZEGEDY C , LIU W , JIA Y Q ,et al. Going deeper with convolutions[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2015: 1-9.
[6]	HE K , ZHANG X , REN S ,et al. Deep residual learning for image recognition[C]// Proceedings of the2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 770-778.
[7]	Huang G , Liu Z , Van Der Maaten L ,et al. Densely connected convolutional networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 4700-4708.
[8]	HU J , SHEN L , SUN G . Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 7132-7141.
[9]	HAN S , POOL J , TRAN J ,et al. Learning both weights and connections for efficient neural networks[J]. arXiv preprint arXiv:1506.02626, 2015.
[10]	HAN S , MAO H , DALLY W J . Deep compression:compressing deep neural networks with pruning,trained quantization and huffman coding[J]. arXiv preprint arXiv:1510.00149, 2015.
[11]	GUO Y , YAO A , CHEN Y . Dynamic network surgery for efficient dnns[J]. arXiv preprint arXiv:1608.04493, 2016.
[12]	MAO H , HAN S , POOL J ,et al. Exploring the regularity of sparse structure in convolutional neural networks[J]. arXiv preprint arXiv:1705.08922, 2017.
[13]	HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network[J]. arXiv preprint arXiv:1503.02531, 2015.
[14]	CHEN J , RAN X . Deep Learning With Edge Computing: AReview[J]. Proceedings of the IEEE, 2019,107(8): 1655-1674.
[15]	LACEY G , TAYLOR G W , AREIBI S . Deep learning on fpgas:Past,present,and future[J]. arXiv preprint arXiv:1602.04283, 2016.
[16]	NING Z , FENG Y , COLLOTTA M ,et al. Deep learning in edge of vehicles:Exploring trirelationship for data transmission[J]. IEEE Transactions on Industrial Informatics, 2019,15(10): 5737-5746.
[17]	IANDOLA F N , HAN S , MOSKEWICZ M W ,et al. SqueezeNet:AlexNet-level accuracy with 50x fewer parameters and<0.5 MB model size[J]. arXiv preprint arXiv:1602.07360, 2016.
[18]	GHOLAMI A , KWON K , WU B ,et al. Squeezenext: Hardware-aware neural network design[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE Press, 2018: 1638-1647.
[19]	ZHANG X , ZHOU X , LIN M ,et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]// Proceedings of the2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 6848-6856.
[20]	HUANG G , LIU S , VAN DER MAATEN L ,et al. Condensenet:an efficient densenet using learned group convolutions[C]// Proceedings of the2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 2752-2761.
[21]	ZOPH B , VASUDEVAN V , SHLENS J ,et al. Learning transferable architectures for scalable image recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 8697-8710.
[22]	LIU C , ZOPH B , NEUMANN M ,et al. Progressive neural architecture search[C]// Proceedings of the European Conference on Computer Vision (ECCV). Piscataway:IEEE Press, 2018: 19-34.
[23]	TAN M , CHEN B , PANG R ,et al. Mnasnet: platform-aware neural architecture search for mobile[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2019: 2820-2828.
[24]	HOWARD A G , ZHU M , CHEN B ,et al. Mobilenets:efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017.
[25]	SANDLER M , HOWARD A , ZHU M ,et al. Mobilenetv2:inverted residuals and linear bottlenecks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4510-4520.
[26]	HOWARD A , SANDLER M , CHU G ,et al. Searching for mobilenetv3[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 1314-1324.
[27]	ZOPH B , LE Q V . Neural architecture search with reinforcement learning[J]. arXiv preprint arXiv:1611.01578, 2016.
[28]	SHI W , CAO J , ZHANG Q ,et al. Edge computing:vision and challenges[J]. IEEE Internet of Things Journal, 2016,3(5): 637-646.
[29]	YANG T J , HOWARD A , CHEN B ,et al. Netadapt: platform-aware neural network adaptation for mobile applications[C]// Proceedings of the European Conference on Computer Vision (ECCV). Piscataway: IEEE Press, 2018: 285-300.
[30]	TEERAPITTAYANON S , MCDANEL B , KUNG H T . Distri buted deep neural networks over the cloud,the edge and end devices[C]// Proceedings of 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS). Piscataway: IEEE Press, 2017: 328-339.
[31]	BONOMI F , MILITO R , ZHU J ,et al. Fog computing and its role in the internet of things[C]// Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing. Piscataway: IEEE Press, 2012: 13-16.
[32]	SATYANARAYANAN M , BAHL P , CACERES R ,et al. The case for VM-based cloudlets in mobile computing[J]. IEEE Pervasive Computing, 2009,8(4): 14-23.
[33]	WANG X , HAN Y , LEUNG V C M ,et al. Convergence of edge computing and deep learning: A comprehensive survey[J]. IEEE Communications Surveys ＆ Tutorials, 2020,22(2): 869-904.
[34]	JEONG H J , LEE H J , SHIN C H ,et al. IONN: incremental offloading of neural network computations from mobile devices to edge servers[C]// Proceedings of the ACM Symposium on Cloud Computing. Piscataway: IEEE Press, 2018: 401-411.
[35]	JEONG H J , LEE H J , SHIN K Y ,et al. PerDNN: offloading deep neural network computations to pervasive edge servers[C]// Proceedings of 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS). Piscataway:IEEE Press, 2020: 1055-1066.
[36]	LI G , LIU L , WANG X ,et al. Auto-tuning neural network quantization framework for collaborative inference between the cloud and edge[C]// Proceedings of International Conference on Artificial Neural Networks. Heidelberg:Springer, 2018: 402-411.
[37]	WU J , LENG C , WANG Y ,et al. Quantized convolutional neural networks for mobile devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2016: 4820-4828.
[38]	TIAN X , ZHU J , XU T ,et al. Mobility-included DNN partition offloading from mobile devices to edge clouds[J]. Sensors, 2021,21(1): 229.
[39]	BOYKOV Y , KOLMOGOROV V . An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision[J]. IEEE Transactions on Pattern Analysis and Computer Vision Intelligence, 2004,26(9): 1124-1137.
[40]	ZHAO Z , BARIJOUGH K M , GERSTLAUER A . Deepthings:distributed adaptive deep learning inference on resource-constrained IoT edge clusters[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018,37(11): 2348-2359.
[41]	HUANG Y , ZHU Y , FAN X ,et al. Task scheduling with optimized transmission time in collaborative cloud-edge learning[C]// Proceedings of 2018 27th International Conference on Computer Communication and Networks (ICCCN). Piscataway:IEEE Press, 2018: 1-9.
[42]	KO J H , NA T , AMIR M F ,et al. Edge-host partitioning of deep neural networks with feature space encoding for resource-constrained internet-of-things platforms[C]// Proceedings of 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). Piscataway: IEEE Press, 2018: 1-6.
[43]	XU S , ZHANG Z , KADOCH M ,et al. A collaborative cloud-edge computing framework in distributed neural network[J]. EURASIP Journal on Wireless Communications and Networking, 2020,2020(1): 1-17.
[44]	WANG W , LAI Q , FU H ,et al. Salient object detection in the deep learning era:an in-depth survey[J]. IEEE Transactions on Pattern Analysis and Computer vision Intelligence, 2021.
[45]	WANG X , GIRSHICK R , GUPTA A ,et al. Non-local neural networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 7794-7803.
[46]	朱应钊, 李嫚 . 元学习研究综述[J]. 电信科学, 2021,37(1): 22-31.
	ZHU Y Z , LI M . Review on meta-learning[J]. Telecommunications Science, 2021,37(1): 22-31.

网络名称	网络深度	参数数量/MB	浮点运算数量/10⁹次	Top1错误率
AlexNet	8	61	1.45	36.7%
VGG16	16	138	31.0	25.6%
Inception-v1	22	5	2.86	30.2%
Inception-v2	42	11	4.0	25.2%
Inception-v3	48	24	11.4	21.2%
Inception-v4	76	35	24.5	19.9%
ResNet-152	152	60	22.6	19.4%
ResNeXt	50	68	8.4	19.1%
DenseNet	201	20	8.4	13.6%