基于标签量信息的联邦学习节点选择算法

doi:10.11959/j.issn.2096-3750.2021.00249

物联网学报 ›› 2021, Vol. 5 ›› Issue (4): 46-53.doi: 10.11959/j.issn.2096-3750.2021.00249

所属专题：联邦学习

基于标签量信息的联邦学习节点选择算法

马嘉华¹, 孙兴华¹, 夏文超², 王玺钧¹, 谭洪舟¹, 朱洪波²

¹ 中山大学，广东广州510006
² 南京邮电大学，江苏南京 210023

修回日期:2021-11-23 出版日期:2021-12-30 发布日期:2021-12-01
作者简介:马嘉华（1997− ），男，中山大学电子与通信工程学院硕士生，主要研究方向为无线通信、联邦学习、人工智能、边缘计算等
孙兴华（1985− ），男，博士，中山大学电子与通信工程学院副教授，主要研究方向为下一代无线通信网络、智能通信等
夏文超（1991− ），男，博士，南京邮电大学副教授，主要研究方向为边缘智能、大规模MIMO、云无线接入网
王玺钧（1984− ），男，博士，中山大学电子与信息工程学院副教授，主要研究方向为信息年龄、强化学习等
谭洪舟（1965− ），男，博士，中山大学电子与信息工程学院教授，主要研究方向为物联网芯片与系统技术
朱洪波（1956− ），男，博士，南京邮电大学教授、博士生导师，南京邮电大学原副校长、物联网研究院院长，江苏省“泛在无线通信与物联网”科技创新团队带头人，主要研究方向为物联网、移动通信网络等
基金资助:
国家重点研发计划(2019YFE0114000);国家自然科学基金资助项目(92067201);江苏省自然科学基金资助项目(BK20212001);广东省基础与应用基础研究基金资助项目(2021A1515012631);广东省基础与应用基础研究基金资助项目(2019A1515011906)

Node selection based on label quantity information in federated learning

Jiahua MA¹, Xinghua SUN¹, Wenchao XIA², Xijun WANG¹, Hongzhou TAN¹, Hongbo ZHU²

¹ Sun Yat-sen University, Guangzhou 510006, China
² Nanjing University of Posts and Telecommunications, Nanjing 210023, China

Revised:2021-11-23 Online:2021-12-30 Published:2021-12-01
Supported by:
The National Key Research and Development Program of China(2019YFE0114000);The National Natural Science Foundation of China(92067201);The Natural Science Foundation of Jiangsu Province(BK20212001);The Guang-dong Basic and Applied Basic Research Foundation(2021A1515012631);The Guang-dong Basic and Applied Basic Research Foundation(2019A1515011906)

摘要/Abstract

摘要：

针对节点数据分布差异给联邦学习算法性能带来不良影响的问题，提出了一个基于标签量信息的节点选择算法。算法设计了一个关于节点标签量信息的优化目标，考虑在一定时耗限制下选择标签分布尽可能均衡的节点组合优化问题。根据节点组合的综合标签分布与模型收敛的相关性，新算法降低了全局模型的权重偏移上界以改善算法的收敛稳定性。仿真验证了新算法与现有的节点选择算法相比拥有更高的收敛效率。

关键词: 联邦学习, 节点选择, 通信时延

Abstract:

Aiming at the problem that the difference of node data distribution has adverse effect on the performance of federated learning algorithm, a node selection algorithm based on label quantity information was proposed.An optimization objective based on the label quantity information of nodes was designed, considering the optimization problem of selecting the nodes with balanced label distribution under a certain time consumption limit.According to the correlation between the aggregated label distribution of selected nodes and the convergence of the global model, the upper bound of the weight divergence of the global model was reduced to improve the convergence stability of the algorithm.Simulation results shows that the new algorithm had higher convergence efficiency than the existing node selection algorithm.

Key words: federated learning, node selection, communication delay

中图分类号:

TP391.4

马嘉华, 孙兴华, 夏文超, 王玺钧, 谭洪舟, 朱洪波. 基于标签量信息的联邦学习节点选择算法[J]. 物联网学报, 2021, 5(4): 46-53.

Jiahua MA, Xinghua SUN, Wenchao XIA, Xijun WANG, Hongzhou TAN, Hongbo ZHU. Node selection based on label quantity information in federated learning[J]. Chinese Journal on Internet of Things, 2021, 5(4): 46-53.

图/表 7

图1

图2

图3

表1

表2

图4

图5

参考文献 16

[1]	MCMAHAN H B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[EB]. 2016.
[2]	YANG H H , LIU Z Z , QUEK T Q S ,et al. Scheduling policies for federated learning in wireless networks[J]. IEEE Transactions on Communications, 2020,68(1): 317-333.
[3]	NISHIO T , YONETANI R . Client selection for federated learning with heterogeneous resources in mobile edge[C]// Proceedings of ICC 2019- 2019 IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2019: 1-7.
[4]	REN J K , HE Y H , WEN D Z ,et al. Scheduling for cellular federated edge learning with importance and channel awareness[J]. IEEE Transactions on Wireless Communications, 2020,19(11): 7690-7703.
[5]	HUANG T S , LIN W W , WU W T ,et al. An efficiency-boosting client selection scheme for federated learning with fairness guarantee[J]. IEEE Transactions on Parallel and Distributed Systems, 2021,32(7): 1552-1564.
[6]	XU J , WANG H Q . Client selection and bandwidth allocation in wireless federated learning networks:a long-term perspective[J]. IEEE Transactions on Wireless Communications, 2021,20(2): 1188-1200.
[7]	SHI W Q , ZHOU S , NIU Z S ,et al. Joint device scheduling and resource allocation for latency constrained wireless federated learning[J]. IEEE Transactions on Wireless Communications, 2021,20(1): 453-467.
[8]	ZHANG W Y , WANG X M , ZHOU P ,et al. Client selection for federated learning with non-IID data in mobile edge computing[J]. IEEE Access, 2021,9: 24462-24474.
[9]	RIBERO M , VIKALO H . Communication-efficient federated learning via optimal client sampling[EB]. 2020.
[10]	WU W T , HE L G , LIN W W ,et al. Accelerating federated learning over reliability-agnostic clients in mobile edge computing systems[J]. IEEE Transactions on Parallel and Distributed Systems, 2021,32(7): 1539-1551.
[11]	NGUYEN H T , SEHWAG V , HOSSEINALIPOUR S ,et al. Fast-convergent federated learning[J]. IEEE Journal on Selected Areas in Communications, 2021,39(1): 201-218.
[12]	YEGANEH Y , FARSHAD A , NAVAB N ,et al. Inverse distance aggregation for federated learning with non-IID data[M]// Domain Adaptation and Representation Transfer,and Distributed and Collaborative Learning. Cham: Springer International Publishing, 2020: 150-159.
[13]	ZHAO Y , LI M , LAI L Z ,et al. Federated learning with non-IID data[EB]. 2018.
[14]	MA J H , SUN X H , XIA W C ,et al. Client selection based on label quantity information for federated learning[C]// PIMRC:Workshop on Native-AI Empowered Wireless Networks. 2021.
[15]	CALDAS S , WU P , LI T ,et al. LEAF:a benchmark for federated settings[EB]. 2018.
[16]	AKDENIZ M R , LIU Y P , SAMIMI M K ,et al. Millimeter wave channel modeling and cellular capacity evaluation[J]. IEEE Journal on Selected Areas in Communications, 2014,32(6): 1164-1179.

算法名	运行回合总数			到达目标识别率耗时
算法名	运行回合总数	50%	60%	70%	80%	85%
FedBag	1 169	3.924 2 h	6.111 9 h	17.152 5 h	31.633 3 h	59.159 9 h
FedCS	1 235	5.038 0 h	9.479 3 h	20.023 8 h	44.540 4 h	78.161 5 h
RS	364	11.506 3 h	19.463 7 h	45.226 8 h	111.395 5 h	207.675 1 h
EM	1 398	3.881 7 h	7.988 1 h	13.445 8 h	42.140 8 h	66.910 1 h

T_limit/s	运行回合总数			到达目标识别率耗时
T_limit/s	运行回合总数	50%	60%	70%	80%	85%
100	2320	2.9230 h	4.3990 h	9.8239 h	21.2913 h	28.4150 h
200	1169	3.9242 h	6.1119 h	17.1525 h	31.6333 h	59.1599 h
300	872	3.3955 h	9.0295 h	16.4829 h	43.4716 h	78.7371 h

基于标签量信息的联邦学习节点选择算法

Node selection based on label quantity information in federated learning

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 16

相关文章 4

Metrics

推荐阅读 0

[1]	耿光磊, 高博, 熊轲, 樊平毅, 陆杨, 王煜炜. 联邦学习赋能6G网络综述[J]. 物联网学报, 2023, 7(2): 50-66.
[2]	郭佳慧, 陈卓越, 高玮, 王玺钧, 孙兴华, 高林. 基于背包模型的联邦学习客户端选择方法[J]. 物联网学报, 2022, 6(4): 158-168.
[3]	郭英芸, 高博, 张志飞, 张煜, 熊轲. 一种基于带宽分配的联邦学习激励机制[J]. 物联网学报, 2022, 6(4): 82-92.
[4]	石光明, 肖泳, 李莹玉, 高大化, 谢雪梅. 面向万物智联的语义通信网络[J]. 物联网学报, 2021, 5(2): 26-36.