通信学报 ›› 2022, Vol. 43 ›› Issue (6): 16-27.doi: 10.11959/j.issn.1000-436x.2022128
• 专题:面向6G的智能至简网络关键技术 • 上一篇 下一篇
王志勤1, 江甲沫1, 刘沛西2, 曹晓雯3, 李阳1, 韩凯峰1, 杜滢1, 朱光旭4
修回日期:
2022-04-27
出版日期:
2022-06-01
发布日期:
2022-06-01
作者简介:
王志勤(1970- ),女,北京人,博士,中国信息通信研究院教授级高级工程师,主要研究方向为无线移动通信技术和标准基金资助:
Zhiqin WANG1, Jiamo JIANG1, Peixi LIU2, Xiaowen CAO3, Yang LI1, Kaifeng HAN1, Ying DU1, Guangxu ZHU4
Revised:
2022-04-27
Online:
2022-06-01
Published:
2022-06-01
Supported by:
摘要:
目的:为充分利用分布在网络边缘的丰富数据使之服务于人工智能模型训练,以联邦边缘学习为代表的边缘智能技术应运而生,本文综述了面向 6G 的联邦边缘学习技术,能够充分利用分布在网络边缘的丰富数据使之服务于人工智能模型训练,并以最优化学习性能(如优化模型训练时间、学习收敛性等)为目标设计无线资源管理策略。
方法:利用联邦学习网络架构,本文对资源分配和用户调度方案进行分析:1.对于资源分配问题分析在以最小化总训练时间为目标时的通信轮数和每轮时延之间的折中关系,为了满足每轮训练的时间约束,应该给计算能力低的设备分配更多的频率带宽,通过减小通信时间补偿计算时间,反之亦然。因此,设备间的带宽分配应该同时考虑信道条件和计算资源,这与传统只考虑信道条件的带宽分配工作截然不同。为此,需要建模总训练时间最小化问题,优化量化级数和带宽分配,设计交替优化算法解决该问题。2.对于用户调度问题通过将数据重要性和仍需通信轮次相联系、信道质量和单轮通信延时相联系,利用理论模型将两者统一,建模通信时间最小化优化问题。通过求解该优化问题发现,最优的调度策略在前期会更多地注重数据重要性,而在后期更注重信道质量。此外,还将所提的单设备调度算法也扩展至多设备调度场景当中。
结果:1.对于资源分配问题,当带宽分配最优时,通过仿真得到的总训练时间与量化级数的关系,在每个量化级数上运行相同的训练过程至少5次,且存在使总训练时间最小化的最优量化级数。总训练时间
结论:1.最优量化级数和最优带宽分配下的训练损失在更短的时间内达到了预定的阈值,并且能取得最高的测试准确率。其次,非最优量化级数和最优带宽分配下的训练性能会优于最优量化级数和平均带宽分配的性能,这同时验证了资源分配的必要性。2.TLM方案在训练早期取得了稍好的性能,充分训练后明显优于所有其他方案。这是由于所提TLM方案中固有的前瞻性质比现有的CA、IA和ICA方案中的近视性质更有优势。
中图分类号:
王志勤, 江甲沫, 刘沛西, 曹晓雯, 李阳, 韩凯峰, 杜滢, 朱光旭. 6G联邦边缘学习新范式:基于任务导向的资源管理策略[J]. 通信学报, 2022, 43(6): 16-27.
Zhiqin WANG, Jiamo JIANG, Peixi LIU, Xiaowen CAO, Yang LI, Kaifeng HAN, Ying DU, Guangxu ZHU. New design paradigm for federated edge learning towards 6G:task-oriented resource management strategies[J]. Journal on Communications, 2022, 43(6): 16-27.
[1] | IMT-2030(6G)推进组. 6G总体愿景与潜在关键技术白皮书[R]. 2021. |
IMT-2030 (6G) Promotion Group. White paper on 6G overall vision and potential key technology[R]. 2021. | |
[2] | 张平, 牛凯, 田辉 ,等. 6G 移动通信技术展望[J]. 通信学报, 2019,40(1): 141-148. |
ZHANG P , NIU K , TIAN H ,et al. Technology prospect of 6G mobile communications[J]. Journal on Communications, 2019,40(1): 141-148. | |
[3] | LETAIEF K B , CHEN W , SHI Y ,et al. The roadmap to 6G:AI empowered wireless networks[J]. IEEE Communication Magazine, 2019,57(8): 84-90. |
[4] | WANG Z Q , DU Y , WEI K J ,et al. Vision,application scenarios,and key technology trends for 6G mobile communications[J]. Science China Information Sciences, 2022,65(5): 1-27. |
[5] | ZHU G X , LIU D Z , DU Y Q ,et al. Toward an intelligent edge:wireless communication meets machine learning[J]. IEEE Communications Magazine, 2020,58(1): 19-25. |
[6] | LI E , ZENG L , ZHOU Z ,et al. Edge AI:On-demand accelerating deep neural network inference via edge computing[J]. IEEE Transaction on Wireless Communication, 2020,19(1): 447-457. |
[7] | ZHU G X , XU J , HUANG K B ,et al. Over-the-air computing for wireless data aggregation in massive IoT[J]. IEEE Wireless Communications, 2021,28(4): 57-65. |
[8] | KONECNY J , MCMAHAN H B , YU F X ,et al. Federated learning:strategies for improving communication efficiency[J]. arXiv Preprint,arXiv:1610.05492, 2016. |
[9] | YANG Q , LIU Y , CHEN T ,et al. Federated machine learning:concept and applications[J]. ACM Transaction on Intelligent Systems, 2019,10(2): 1-19. |
[10] | CHEN M , YANG Z , SAAD W ,et al. A joint learning and communications framework for federated learning over wireless networks[J]. IEEE Transactions on Wireless Communications, 2020,20(1): 269-283. |
[11] | DAYAN I , ROTH H R , ZHONG A X ,et al. Federated learning for predicting clinical outcomes in patients with COVID-19[J]. Nature Medicine, 2021,27(10): 1735-1743. |
[12] | SAVAZZI S , NICOLI M , BENNIS M ,et al. Opportunities of federated learning in connected,cooperative,and automated industrial systems[J]. IEEE Communications Magazine, 2021,59(2): 16-21. |
[13] | MILLS J , HU J , MIN G Y . Communication-efficient federated learning for wireless edge intelligence in IoT[J]. IEEE Internet of Things Journal, 2020,7(7): 5986-5994. |
[14] | LAN Q , WEN D Z , ZHANG Z Z ,et al. What is semantic communication? A view on conveying meaning in the era of machine intelligence[J]. arXiv Preprint,arXiv:2110.00196, 2021. |
[15] | TSE D , VISWANATH P . Fundamentals of wireless communication[M]. Cambridge: Cambridge University Press, 2005. |
[16] | SHEN C , XU J , ZHENG S H ,et al. Resource rationing for wireless federated learning:concept,benefits,and challenges[J]. IEEE Communications Magazine, 2021,59(5): 82-87. |
[17] | CHEN M Z , YANG Z H , SAAD W ,et al. A joint learning and communications framework for federated learning over wireless networks[J]. IEEE Transactions on Wireless Communications, 2021,20(1): 269-283. |
[18] | WANG Y M , XU Y Q , SHI Q J ,et al. Quantized federated learning under transmission delay and outage constraints[J]. IEEE Journal on Selected Areas in Communications, 2022,40(1): 323-341. |
[19] | ZHU G X , WANG Y , HUANG K B . Broadband analog aggregation for low-latency federated edge learning[J]. IEEE Transactions on Wireless Communications, 2020,19(1): 491-506. |
[20] | LUO R Y , TIAN H , NI W L . Communication-aware path design for indoor robots exploiting federated deep reinforcement learning[C]// Proceedings of 2021 IEEE 32nd Annual International Symposium on Personal,Indoor and Mobile Radio Communications. Piscataway:IEEE Press, 2021: 1197-1202. |
[21] | WAN S , LU J X , FAN P Y ,et al. Convergence analysis and system design for federated learning over wireless networks[J]. IEEE Journal on Selected Areas in Communications, 2021,39(12): 3622-3639. |
[22] | CHEN M Z , POOR H V , SAAD W ,et al. Convergence time optimization for federated learning over wireless networks[J]. IEEE Transactions on Wireless Communications, 2021,20(4): 2457-2471. |
[23] | REN J Y , SUN J S , TIAN H ,et al. Joint resource allocation for efficient federated learning in Internet of Things supported by edge computing[C]// Proceedings of 2021 IEEE International Conference on Communications Workshops. Piscataway:IEEE Press, 2021: 1-6. |
[24] | MCMAHAN B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[C]// Artificial Intelligence and Statistics. New York:PMLR, 2017: 1273-1282. |
[25] | YANG H H , LIU Z Z , QUEK T Q S ,et al. Scheduling policies for federated learning in wireless networks[J]. IEEE Transactions on Communications, 2020,68(1): 317-333. |
[26] | TAIK A , MLIKA Z , CHERKAOUI S . Data-aware device scheduling for federated edge learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2022,8(1): 408-421. |
[27] | WANG S Q , TUOR T , SALONIDIS T ,et al. Adaptive federated learning in resource constrained edge computing systems[J]. IEEE Journal on Selected Areas in Communications, 2019,37(6): 1205-1221. |
[28] | RIZK E , VLASKI S , SAYED A H . Optimal importance sampling for federated learning[C]// Proceedings of 2021 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2021: 3095-3099. |
[29] | AMIRI M M , GüNDüZ D , . Federated learning over wireless fading channels[J]. IEEE Transactions on Wireless Communications, 2020,19(5): 3546-3557. |
[30] | KONECNY J , MCMAHAN H B , YU F X ,et al. Federated learning:strategies for improving communication and efficiency[J]. arXiv Preprint,arXiv:1610.05492, 2016. |
[31] | AMIRI M M , GUNDUZ D , KULKARNI S R ,et al. Federated learning with quantized global model updates[J]. arXiv Preprint,arXiv:2006.10672, 2020. |
[32] | NORI M K , YUN S , KIM I M . Fast federated learning by balancing communication trade-offs[J]. IEEE Transactions on Communications, 2021,69(8): 5168-5182. |
[33] | 唐伦, 汪智平, 蒲昊 ,等. 基于自适应梯度压缩的高效联邦学习通信机制研究[J]. 电子与信息学报, 2022:doi.org/10.11999/JEIT211262. |
TANG L , WANG Z P , PU H ,et al. Research on efficient federated learning communication mechanism based on adaptive gradient compression[J]. Journal of Electronics & Information Technology, 2022:doi.org/10.11999/JEIT211262. | |
[34] | BOUZINIS P S , DIAMANTOULAKIS P D , KARAGIANNIDIS G K . Wireless quantized federated learning:a joint computation and communication design[J]. arXiv Preprint,arXiv:2203.05878, 2022. |
[35] | ZHU G X , DU Y Q , GüNDüZ D , ,et al. One-bit over-the-air aggregation for communication-efficient federated edge learning:design and convergence analysis[J]. IEEE Transactions on Wireless Communications, 2021,20(3): 2120-2135. |
[36] | ALISTARH D , GRUBIC D , LI J ,et al. QSGD:communication-efficient SGD via gradient quantization and encoding[C]// Advances in Neural Information Process.Systems. Massachusetts:MIT Press, 2017: 1707-1718. |
[37] | LIU P X , JIANG J M , ZHU G X ,et al. Training time minimization for federated edge learning with optimized gradient quantization and bandwidth allocation[J]. arXiv Preprint,arXiv:2112.14387, 2021. |
[38] | RAZAVIYAYN M . Successive convex approximation:analysis and applications[D]. TwenCities:University of Minnesota, 2014. |
[39] | AMIRI M M , GüNDüZ D , KULKARNI S R ,et al. Convergence of update aware device scheduling for federated learning at the wireless edge[J]. IEEE Transactions on Wireless Communications, 2021,20(6): 3643-3658. |
[40] | REN J K , HE Y H , WEN D Z ,et al. Scheduling for cellular federated edge learning with importance and channel awareness[J]. IEEE Transactions on Wireless Communications, 2020,19(11): 7690-7703. |
[41] | YANG H H , ARAFA A , QUEK T Q S ,et al. Age-based scheduling policy for federated learning in mobile edge networks[C]// Proceedings of 2020 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2020: 8743-8747. |
[42] | NISHIO T , YONETANI R . Client selection for federated learning with heterogeneous resources in mobile edge[C]// Proceedings of 2019 IEEE International Conference on Communications. Piscataway:IEEE Press, 2019: 1-7. |
[43] | 贺文晨, 郭少勇, 邱雪松 ,等. 基于DRL的联邦学习节点选择方法[J]. 通信学报, 2021,42(6): 62-71. |
HE W , GUO S , QIU X ,et al. Node selection method in federated learning based on deep reinforcement learning[J]. Journal on Communications, 2021,42(6): 62-71. | |
[44] | ZHANG M J , ZHU G X , WANG S ,et al. Accelerating federated edge learning via optimized probabilistic device scheduling[C]// Proceedings of 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications. Piscataway:IEEE Press, 2021: 606-610. |
[45] | DOSOVITSKIY A , ROS G , CODEVILLA F ,et al. CARLA:an open urban driving simulator[J]. arXiv Preprint,arXiv:1711.03938, 2017. |
[46] | YAN Y , MAO Y X , LI B . SECOND:sparsely embedded convolutional detection[J]. Sensors, 2018,18(10): 3337. |
[47] | ZHANG Z J , WANG S , HONG Y C ,et al. Distributed dynamic map fusion via federated learning for intelligent networked vehicles[C]// Proceedings of 2021 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2021: 953-959. |
[48] | ABARI O , RAHUL H , KATABI D . Over-the-air function computation in sensor networks[J]. arXiv Preprint,arXiv:1612.02307, 2016. |
[49] | JEON S , WANG C , GASTPAR M . Computation over Gaussian networks with orthogonal components[J]. IEEE Transaction on Information Theory, 2014,60(12): 7841-7861. |
[50] | YANG K , JIANG T , SHI Y M ,et al. Federated learning via over-the-air computation[J]. IEEE Transactions on Wireless Communications, 2020,19(3): 2022-2035. |
[51] | IMT-2030(6G)推进组. 通信感知一体化技术研究报告[R]. 2021. |
IMT-2030 (6G) Promotion Group. Research report on integrated sensing and communications technology[R]. 2021. |
[1] | 杨龙, 赵丽, 周雨晨, 贺冰涛, 陈健. 缓存辅助的协作NOMA携能传输[J]. 通信学报, 2023, 44(6): 77-89. |
[2] | 王再见, 谷慧敏. 基于联合优化的网络切片资源分配策略[J]. 通信学报, 2023, 44(5): 234-245. |
[3] | 苏新, 张桂福, 行鸿彦, Zenghui Wang. 基于平衡生成对抗网络的海洋气象传感网入侵检测研究[J]. 通信学报, 2023, 44(4): 124-136. |
[4] | 蒋丽, 谢胜利, 田辉. 面向数字孪生边缘网络的区块链分片及资源自适应优化机制[J]. 通信学报, 2023, 44(3): 12-23. |
[5] | 余雪勇, 邱礼翔, 宋家宁, 朱洪波. 无人机辅助边缘计算中安全通信与能效优化策略[J]. 通信学报, 2023, 44(3): 45-54. |
[6] | 江沸菠, 彭于波, 董莉. 面向6G的深度图像语义通信模型[J]. 通信学报, 2023, 44(3): 198-208. |
[7] | 龙隆, 刘子辰, 陆在旺, 张玉成, 李蕾. 移动边缘网络下服务缓存与资源分配联合优化策略[J]. 通信学报, 2023, 44(1): 64-74. |
[8] | 廖勇, 程港, 李玉杰. 基于深度展开的大规模MIMO系统CSI反馈算法[J]. 通信学报, 2022, 43(12): 77-88. |
[9] | 辜方林, 黄育侦, 赵莺, 赵海涛, 魏急波. 基于单载波频域均衡的直接序列扩频自适应抗干扰技术[J]. 通信学报, 2022, 43(11): 26-34. |
[10] | 陈赓, 齐书虎, 沈斐, 曾庆田. 面向B5G多业务场景基于D3QN的双时间尺度网络切片算法[J]. 通信学报, 2022, 43(11): 213-224. |
[11] | 丛玉良, 孙闻晞, 薛科, 钱志鸿, 陈绵书. 基于改进的混合遗传算法的车联网任务卸载策略研究[J]. 通信学报, 2022, 43(10): 77-85. |
[12] | 苏新, 孟蕾蕾, 周一青, CELIMUGE Wu. 基于深度强化学习的海洋移动边缘计算卸载方法[J]. 通信学报, 2022, 43(10): 133-145. |
[13] | 刘彩霞, 季新生, 邬江兴. 移动通信网的内生安全共性问题及破解之道[J]. 通信学报, 2022, 43(9): 70-79. |
[14] | 朱晓荣, 陈康. 基于细粒度切片的6G网络弹性切换算法研究[J]. 通信学报, 2022, 43(9): 148-156. |
[15] | 许文俊, 吴思雷, 王凤玉, 林兰, 李国军, 张治. 基于多智能体强化学习的大规模灾后用户分布式覆盖优化[J]. 通信学报, 2022, 43(8): 1-16. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||
|