Chinese Journal on Internet of Things ›› 2022, Vol. 6 ›› Issue (4): 72-81.doi: 10.11959/j.issn.2096-3750.2022.00308

• Theory and Technology • Previous Articles     Next Articles

Cooperative inference analysis based on DNN convolutional kernel partitioning

Jialin ZHI, Yinglei TENG, Xinyang ZHANG, Tao NIU, Mei SONG   

  1. School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • Revised:2022-11-07 Online:2022-12-30 Published:2022-12-01
  • Supported by:
    The National Key Research and Development Program of China(2021YFB3300100);The National Natural Science Foundation of China(62171062)

Abstract:

With the popularity of intelligent chip in the application of edge terminal devices, a large number of AI applications will be deployed on the edge of networks closer to data sources in the future.The method based on DNN partition can realize deep learning model training and deployment on resource-constrained terminal devices, and solve the bottleneck problem of edge AI computing ability.Thekernel based partition method (KPM) was proposed as a new scheme on the basis of traditional workload based partition method (WPM).The quantitative analysis of inference performance was carried out from three aspects of computation FLOPS, memory consumption and communication cost respectively, and the qualitative analysis of the above two schemes was carried out from the perspective of flexibility, robustness and privacy of inference process.Finally, a software and hardware experimental platform was built, and AlexNet and VGG11 networks were implemented using PyTorch to further verify the performance advantages of the proposed scheme in terms of delay and energy consumption.It was concluded that, compared with the WPM scheme, the KPM scheme had better DNN reasoning acceleration effect in large-scale computing scenarios.And it has lower memory usage and energy consumption.

Key words: edge intelligence, deep neural network partition, cooperative computation, parallel partition

CLC Number: 

No Suggested Reading articles found!