面向异构流式数据的高性能联邦持续学习算法

doi:10.11959/j.issn.1000-436x.2023102

Abstract

Abstract:

Aiming at the problems of poor model performance and low training efficiency in training streaming data of AI models that provide intelligent services, a high-performance federated continual learning algorithm for heterogeneous streaming data (FCL-HSD) was proposed in the distributed terminal system with privacy data.In order to solve the problem of the current model forgetting old data, a model with dynamically extensible structure was introduced in the local training stage, and an extension audit mechanism was designed to ensure the capability of the AI model to recognize old data at the cost of small storage overhead.Considering the heterogeneity of terminal data, a customized global model strategy based on data distribution similarity was designed at the central server side, and an aggregation-by-block manner was implemented for different modules of the model.The feasibility and effectiveness of the proposed algorithm were verified under various data increment scenarios with different data sets.Experimental results show that, compared with existing works, the proposed algorithm can effectively improve the model performance to classify old data on the premise of ensuring the capability to classify new data.

Key words: heterogeneous data, streaming data, federated learning, federated continual learning, catastrophic forgetting

CLC Number:

TP302

Hui JIANG, Tianliu HE, Min LIU, Sheng SUN, Yuwei WANG. High-performance federated continual learning algorithm for heterogeneous streaming data[J]. Journal on Communications, 2023, 44(5): 123-136.

Figures/Tables 11

References 24

[1]	麻省理工科技评论. 2021 年中国数字经济时代人工智能生态白皮书[R]. 2022.
	MIT Technology Review. 2021 white paper on artificial intelligence ecology in China's digital economy era[R]. 2022.
[2]	BILOGREVIC I , JADLIWALA M , KALKAN K ,et al. Privacy in mobile computing for location-sharing-based services[C]// Inter national Symposium on Privacy Enhancing Technologies Symposium. Berlin:Springer, 2011: 77-96.
[3]	LIANG X H , LI X , LUAN T H ,et al. Morality-driven data forwarding with privacy preservation in mobile social networks[J]. IEEE Transactions on Vehicular Technology, 2012,61(7): 3209-3222.
[4]	KONE?NY J , MCMAHAN H B , RAMAGE D ,et al. Federated optimization:distributed machine learning for on-device intelligence[J]. arXiv Preprint,arXiv:1610.02527, 2016.
[5]	LE J Q , LEI X Y , MU N K ,et al. Federated continuous learning with broad network architecture[J]. IEEE Transactions on Cybernetics, 2021,51(8): 3874-3888.
[6]	SERRA J , SURIS D , MIRON M ,et al. Overcoming catastrophic forgetting with hard attention to the task[C]// Proceedings of the International Conference on Machine Learning. New York:ACM Press, 2018: 4548-4557.
[7]	WU Y , CHEN Y P , WANG L J ,et al. Large scale incremental learning[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2020: 374-382.
[8]	HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016: 770-778.
[9]	CASTRO F M , MARíN-JIMéNEZ M J , GUIL N ,et al. End-to-end incremental learning[C]// European Conference on Computer Vision. Berlin:Springer, 2018: 241-257.
[10]	MASANA M , LIU X L , TWARDOWSKI B ,et al. Class-incremental learning:survey and performance evaluation on image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023,45(5): 5513-5533.
[11]	LI Z Z , HOIEM D . Learning without forgetting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018,40(12): 2935-2947.
[12]	REBUFFI S A , KOLESNIKOV A , SPERL G ,et al. iCaRL:incremental classifier and representation learning[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2017: 5533-5542.
[13]	YAN S P , XIE J W , HE X M . D:dynamically expandable representation for class incremental learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2021: 3013-3022.
[14]	MUN H , LEE Y . Internet traffic classification with federated learning[J]. Electronics, 2020,10(1): 27.
[15]	DONG J H , WANG L X , FANG Z ,et al. Federated class-incremental learning[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2022: 10154-10163.
[16]	LI T , SANJABI M , BEIRAMI A ,et al. Fair resource allocation in federated learning[J]. arXiv Preprint,arXiv:1905.10497, 2019.
[17]	ABAD M S H , OZFATURA E , GUNDUZ D ,et al. Hierarchical federated learning ACROSS heterogeneous cellular networks[C]// Proceed ings of 2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP). Piscataway:IEEE Press, 2020: 8866-8870.
[18]	LONG G , XIE M , SHEN T ,et al. Multi-center federated learning:clients clustering for better personalization[J]. arXiv Preprint,arXiv:2005.01026, 2020.
[19]	KRIZHEVSKY A , SUTSKEVER I , GEOFFREY E H . Learning multiple layers of features from tiny images[J]. Communications of the ACM, 2012,60(6): 84-90.
[20]	DRAPER-GIL G , LASHKARI A H , MAMUN M S I ,et al. Characterization of encrypted and VPN traffic using time-related features[C]// Pro ceedings of the 2nd International Conference on Information Systems Security and Privacy. [S.l]:Scite Press, 2016: 407-414.
[21]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[22]	骆子铭, 许书彬, 刘晓东 . 基于机器学习的TLS恶意加密流量检测方案[J]. 网络与信息安全学报, 2020,60(1): 77-83.
	LUO Z M , XU S B , LIU X D . Scheme for identifying malware traffic with TLS data based on machine learning[J]. Chinese Journal of Network and Information Security, 2020,60(1): 77-83.
[23]	PACHECO F , EXPOSITO E , GINESTE M ,et al. Towards the deployment of machine learning solutions in network traffic classification:a systematic survey[J]. IEEE Communications Surveys ＆ Tutorials, 2019,21(2): 1988-2014.
[24]	XIE G R , LI Q , JIANG Y . Self-attentive deep learning method for online traffic classification and its interpretability[J]. Computer Networks, 2021,196:108267.

Metrics

Recommended 0

No Suggested Reading articles found!

High-performance federated continual learning algorithm for heterogeneous streaming data

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 24

Related Articles 15

Metrics

Recommended 0

[1]	Xindi MA, Qinghua LI, Qi JIANG, Zhuo MA, Sheng GAO, Youliang TIAN, Jianfeng MA. Byzantine-robust federated learning over Non-IID data [J]. Journal on Communications, 2023, 44(6): 138-153.
[2]	Kaiju LI, Qiang XU, Hao WANG. Communication-efficient federated learning method via redundant data elimination [J]. Journal on Communications, 2023, 44(5): 79-93.
[3]	Shengxing YU, Zekai CHEN, Zhong CHEN, Ximeng LIU. DAGUARD: distributed backdoor attack defense scheme under federated learning [J]. Journal on Communications, 2023, 44(5): 110-122.
[4]	Youliang TIAN, Shihong WU, Ta LI, Lindong WANG, Hua ZHOU. Federated learning optimization algorithm based on incentive mechanism [J]. Journal on Communications, 2023, 44(5): 169-180.
[5]	Jiale ZHANG, Chengcheng ZHU, Xiaobing SUN, Bing CHEN. Membership inference attack and defense method in federated learning based on GAN [J]. Journal on Communications, 2023, 44(5): 193-205.
[6]	Shengxing YU, Zhong CHEN. Efficient secure federated learning aggregation framework based on homomorphic encryption [J]. Journal on Communications, 2023, 44(1): 14-28.
[7]	Lingtao TANG, Di WANG, Shengyun LIU. Data augmentation scheme for federated learning with non-IID data [J]. Journal on Communications, 2023, 44(1): 164-176.
[8]	Shaoshuai FAN, Jianbo WU, Hui TIAN. Federated learning resource management for energy-constrained industrial IoT devices [J]. Journal on Communications, 2022, 43(8): 65-77.
[9]	Zijia MO, Zhipeng GAO, Yang YANG, Yijing LIN, Shan SUN, Chen ZHAO. Efficient distributed model sharing strategy for data privacy protection in Internet of vehicles [J]. Journal on Communications, 2022, 43(4): 83-94.
[10]	Xiaoyan ZHANG, Zhihao LIU, Xiaofeng DU, Tianbo LU. Research on a real-time receiving scheme of streaming data [J]. Journal on Communications, 2022, 43(4): 154-163.
[11]	Haiyan KANG, Yuanrui JI. Research on federated learning approach based on local differential privacy [J]. Journal on Communications, 2022, 43(10): 94-105.
[12]	Meixia TAO, Dong WANG, Rui SUN, Naifu ZHANG. TDMA-based user scheduling policies for federated learning [J]. Journal on Communications, 2021, 42(6): 1-29.
[13]	Wenchen HE, Shaoyong GUO, Xuesong QIU, Liandong CHEN, Suxiang ZHANG. Node selection method in federated learning based on deep reinforcement learning [J]. Journal on Communications, 2021, 42(6): 62-71.
[14]	Youhuizi LI, Yuyu YIN, Honghao GAO, Yi JIN, Xinheng WANG. Survey on privacy protection in non-aggregated data sharing [J]. Journal on Communications, 2021, 42(6): 195-212.
[15]	Yongming HUANG, Chong ZHENG, Zhengming ZHANG, Xiaohu YOU. Research on mobile edge computing and caching in massive wireless communication network [J]. Journal on Communications, 2021, 42(4): 44-61.