联邦推荐系统综述

doi:10.11959/j.issn.2096-0271.2022032

大数据 ›› 2022, Vol. 8 ›› Issue (4): 105-132.doi: 10.11959/j.issn.2096-0271.2022032

联邦推荐系统综述

朱智韬¹^,², 司世景¹, 王健宗¹, 肖京¹

¹ 平安科技（深圳）有限公司，广东深圳 518063
² 中国科学技术大学，安徽合肥 230026

出版日期:2022-07-15 发布日期:2022-07-01
作者简介:朱智韬（1996- ），男，中国科学技术大学硕士生，平安科技（深圳）有限公司算法工程师，中国计算机学会（CCF）会员，主要研究方向为人工智能、联邦学习和推荐系统等
司世景（1988- ），男，博士，平安科技（深圳）有限公司资深算法研究员，中国科学技术大学硕士生企业导师， CCF会员。发表机器学习、大数据和人工智能领域国际核心论文20余篇
王健宗（1983- ），男，博士，平安科技（深圳）有限公司副总工程师、资深人工智能总监。CCF理事、杰出会员， CCF大数据专家委员会委员，主要研究方向为联邦学习、深度学习、云计算、物联网和元宇宙
肖京（1972- ），男，博士，平安科技（深圳）有限公司首席科学家，深圳市政协委员，中国计算机学会深圳会员活动中心副主席，清华大学、上海交通大学、同济大学、香港中文大学、深圳大学、上海纽约大学客座教授，长期从事人工智能与大数据分析挖掘相关领域研究工作，发表计算机图形学、自动驾驶、3D显示、医疗诊断、联邦学习等领域国际核心论文230余篇，授权专利220余项
基金资助:
广东省重点领域研发计划“新一代人工智能”重大专项(2021B0101400003)

Survey on federated recommendation systems

Zhitao ZHU¹^,², Shijing SI¹, Jianzong WANG¹, Jing XIAO¹

¹ Ping An Technology (Shenzhen) Co., Ltd., Shenzhen 518063, China
² University of Science and Technology of China, Hefei 230026, China

Online:2022-07-15 Published:2022-07-01
Supported by:
The Key Research and Development Program of Guangdong Province(2021B0101400003)

摘要/Abstract

摘要：

在联邦学习范式中，原始数据被本地存储在独立的用户客户端中，而脱敏数据被发送到中心服务器中加以聚合，这给众多领域提供了一种新颖的设计思路。考虑到传统推荐系统的研究方向集中于提高推荐效果，在资源节约、跨领域推荐、隐私保护等方面还具有很大改进空间，如何将联邦学习与推荐系统结合以解决这些问题成为当前的一个研究热点。对近年来基于联邦学习的推荐系统进行了全面的总结、比较与分析，首先介绍了推荐系统的传统实现方式及面临的瓶颈；然后引入了联邦学习范式，描述了联邦学习在隐私保护、利用多领域用户数据两方面给推荐系统带来的增益，以及二者结合的技术挑战，进而详细说明了现有的联邦推荐系统部署方式；最后，对联邦推荐系统未来的研究进行了展望与总结。

关键词: 联邦学习, 推荐系统, 隐私保护, 协同过滤, 深度学习

Abstract:

In the federated learning (FL) paradigm, the original data are stored in independent clients while masked data are sent to a central server to be aggregated, which proposes a novel design approach to numerous domains.Given the wide application of recommendation systems (RS) in diverse domains, combining RS with FL techniques has been gaining momentum to reduce the computational cost, do cross-domain recommendation and protect users’ privacy while maintaining recommendations performance as traditional RS.The federated learning-based recommendation systems in recent years were comprehensively summarized.The difference between traditional and federated recommendation systems was analyzed, and the main research direction and progress of federated recommendation systems were demonstrated with comparison and analysis.Firstly, the traditional recommendation systems and their bottleneck were summarized.Then the federated learning paradigm was introduced.Furthermore, the advantages of combining federated learning with recommendation systems were depicted in two aspects: privacy protection and usage of multi-domain user information, along with the technical challenges during the combination.At the same time, the existing deployment of federated recommendation systems was illustrated in detail.Finally, future research on federated recommendation systems was prospected and summarized.

Key words: federated learning, recommendation system, privacy-preserving, collaborative filtering, deep learning

中图分类号:

TP391

朱智韬, 司世景, 王健宗, 肖京. 联邦推荐系统综述[J]. 大数据, 2022, 8(4): 105-132.

Zhitao ZHU, Shijing SI, Jianzong WANG, Jing XIAO. Survey on federated recommendation systems[J]. Big Data Research, 2022, 8(4): 105-132.

图/表 10

表1

相似度度量方法"

相似度度量	表达式	优点	缺点
闵可夫斯基距离	$dist (X, Y) = {(\sum_{i = 1}^{n} \| x_{i} - y_{i} \|^{p})}^{\frac{1}{p}}$	反映个别数值特征的绝对差异	受离群值影响较大
杰卡德相似系数	$J (A, B) = \frac{\| A \cap B \|}{\| A \cup B \|}$	计算简单，算力负担小	没有考虑用户之间的评分差异；精度不高
余弦相似度	$\cos (θ) = \frac{a^{⊤} b}{\| a \| \cdot \| b \|}$	对绝对值不敏感，修正了用户之间可能存在的参数差异	将缺失值设置为0会缩小真实差距
皮尔逊相关系数	$ρ_{X Y} = \frac{\sum_{i = 1}^{n} (X - μ_{X}) (Y - μ_{Y})}{\sqrt{\sum_{i = 1}^{n} (X - μ_{X})^{2}} \sqrt{\sum_{i = 1}^{n} (Y - μ_{Y})^{2}}}$	用平均值填充缺失的值，以避免分数膨胀	不能直接反映所使用的物品数量；在稀疏的数据集上不易计算相似度

表1

图1

图2

图3

表2

联邦学习分类"

分类	表示
横向联邦学习	$X_{i} = X_{j}, Y_{i} = Y_{j}, I_{i} \neq I_{j} \forall D_{i}, D_{j}, i \neq j$
纵向联邦学习	$X_{i} \neq X_{j}, Y_{i} \neq Y_{j}, I_{i} = I_{j} \forall D_{i}, D_{j}, i \neq j$
联邦迁移学习	$X_{i} \neq X_{j}, Y_{i} \neq Y_{j}, I_{i} \neq I_{j} \forall D_{i}, D_{j}, i \neq j$

表2

图4

表3

图5

图6

表4

参考文献 127

[1]	ZHANG S , YAO L N , SUN A X ,et al. Deep learning based recommender system[J]. ACM Computing Surveys, 2020,52(1): 1-38.
[2]	RICCI F , ROKACH L , SHAPIRA B . Introduction to recommender systems handbook[M]// Recommender Systems Handbook. Heidelberg: Springer, 2011.
[3]	ALBRECHT J P . How the GDPR will change the world[J]. European Data Protection Law Review, 2016,2(3): 287-289.
[4]	MOONEY R J , ROY L . Content-based book recommending using learning for text categorization[C]// Proceedings of the 5th ACM Conference on Digital Libraries. New York:ACM Press, 2000: 195-204.
[5]	SHARDANAND U , MAES P . Social information filtering:algorithms for automating “word of mouth”[C]// Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York:ACM Press, 1995: 210-217.
[6]	MILLER B N , ALBERT I , LAM S K ,et al. MovieLens unplugged:experiences with an occasionally connected recommender system[C]// Proceedings of the 8th International Conference on Intelligent User Interfaces. New York:ACM Press, 2003: 263-266.
[7]	TERVEEN L , HILL W , AMENTO B ,et al. Phoaks:a system for sharing recommendations[J]. Communications of the ACM, 1997,40(3): 59-62.
[8]	GOLDBERG K , ROEDER T , GUPTA D ,et al. Eigentaste:a constant time collaborative filtering algorithm[J]. Information Retrieval, 2001,4(2): 133-151.
[9]	BILLSUS D , PAZZANI M J . Learning collaborative information filters[C]// Proceedings of the 15th International Conference on Machine Learning.San Francisco:Morgan Kaufmann Publishers Inc. , 1998: 46-54.
[10]	ADOMAVICIUS G , TUZHILIN A . Toward the next generation of recommender systems:a survey of the state-of-theart and possible extensions[J]. IEEE Transactions on Knowledge and Data Engineering, 2005,17(6): 734-749.
[11]	DESHPANDE M , KARYPIS G . Item-based top-N recommendation algorithms[J]. ACM Transactions on Information Systems, 2004,22(1): 143-177.
[12]	SU X Y , KHOSHGOFTAAR T M . A survey of collaborative filtering techniques[J]. Advances in Artificial Intelligence, 2009:421425.
[13]	CHOI K , YOO D , KIM G ,et al. A hybrid online-product recommendation system:combining implicit rating-based collaborative filtering and sequential pattern analysis[J]. Electronic Commerce Research and Applications, 2012,11(4): 309-317.
[14]	XIANG L , YUAN Q , ZHAO S W ,et al. Temporal recommendation on graphs via long- and short-term preference fusion[C]// Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2010: 723-732.
[15]	BREESE J S , HECKERMAN D , KADIE C . Empirical analysis of predictive algorithms for collaborative filtering[C]// Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence.San Francisco:Morgan Kaufmann Publishers Inc. , 1998: 43-52.
[16]	UNGAR L H , FOSTER D P . Clustering methods for collaborative filtering[C]// Proceedings of 1998 AAAI Workshop on Recommendation Systems. Palo Alto:AAAI Press, 1998: 114-129.
[17]	黄立威, 江碧涛, 吕守业 ,等. 基于深度学习的推荐系统研究综述[J]. 计算机学报, 2018,41(7): 1619-1647.
	HUANG L W , JIANG B T , LYU S Y ,et al. Survey on deep learning based recommender systems[J]. Chinese Journal of Computers, 2018,41(7): 1619-1647.
[18]	PAZZANI M J , BILLSUS D . Contentbased recommendation systems[M]// The adaptive Web. Heidelberg: Springer, 2007.
[19]	SALTON G , WONG A , YANG C S . A vector space model for automatic indexing[J]. Communications of the ACM, 1975,18(11): 613-620.
[20]	BURKE R . Hybrid recommender systems:survey and experiments[J]. User Modeling and User-Adapted Interaction, 2002,12(4): 331-370.
[21]	WANG H , WANG N Y , YEUNG D Y . Collaborative deep learning for recommender systems[C]// Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2015: 1235-1244.
[22]	YANG X W , STECK H , LIU Y . Circlebased recommendation in online social networks[C]// Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2012: 1267-1275.
[23]	KRULWICH B . LIFESTYLE FINDER:intelligent user profiling using large-scale demographic data[J]. AI Magazine, 1997,18(2): 37-45.
[24]	GONZáLEZ G , LOPEZ B , ROSA J L . A multi-agent smart user model for cross-domain recommender systems[C]// Proceedings of 2005 International Conference on Intelligent User Interfaces:Beyond Personalization. New York:ACM Press, 2005.
[25]	DENG L . Deep learning:methods and applications[J]. Foundations and Trends in Signal Processing, 2014,7(3/4): 197-387.
[26]	王健宗, 黄章成, 肖京 . 人工智能赋能金融科技[J]. 大数据, 2018,4(3): 111-116.
	WANG J Z , HUANG Z C , XIAO J . Artificial intelligence energize Fintech[J]. Big Data Research, 2018,4(3): 111-116.
[27]	SALAKHUTDINOV R , MNIH A , HINTON G . Restricted Boltzmann machines for collaborative filtering[C]// Proceedings of the 24th International Conference on Machine Learning. New York:ACM Press, 2007: 791-798.
[28]	SEDHAIN S , MENON A K , SANNER S ,et al. AutoRec:autoencoders meet collaborative filtering[C]// Proceedings of the 24th International Conference on World Wide Web. New York:ACM Press, 2015: 111-112.
[29]	HE X N , LIAO L Z , ZHANG H W ,et al. Neural collaborative filtering[C]// Proceedings of the 26th International Conference on World Wide Web. New York:ACM Press, 2017: 173-182.
[30]	ZHOU G R , ZHU X Q , SONG C R ,et al. Deep interest network for click-through rate prediction[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining. New York:ACM Press, 2018: 1059-1068.
[31]	ZHOU G R , MOU N , FAN Y ,et al. Deep interest evolution network for clickthrough rate prediction[C]// Proceedings of 2019 AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2019: 5941-5948.
[32]	LI C , LIU Z Y , WU M M ,et al. Multiinterest network with dynamic routing for recommendation at Tmall[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York:ACM Press, 2019: 2615-2623.
[33]	HE X N , CHUA T S . Neural factorization machines for sparse predictive analytics[C]// Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2017: 355-364.
[34]	GUO H F , TANG R M , YE Y M ,et al. DeepFM:a factorization-machine based neural network for CTR prediction[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2017: 1725-1731.
[35]	XIAO J , YE H , HE X N ,et al. Attentional factorization machines:learning the weight of feature interactions via attention networks[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2017: 3119-3125.
[36]	GONG Y Y , ZHANG Q . Hashtag recommendation using attentionbased convolutional neural network[C]// Proceedings of the 25th International Joint Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2016: 2782-2788.
[37]	ZHANG Q , WANG J W , HUANG H R ,et al. Hashtag recommendation for multimodal microblog using coattention network[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2017: 3420-3426.
[38]	OORD A V D , DIELEMAN S , SCHRAUWEN B . Deep content-based music recommendation[C]// Proceedings of the 26th International Conference on Nerual Information Processing Systems.Red Hook:Curran Associates Inc. , 2013: 2643-2651.
[39]	LEI C Y , LIU D , LI W P ,et al. Comparative deep learning of hybrid representations for image recommendations[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 2545-2553.
[40]	HIDASI B A Z , KARATZOGLOU A , BALTRUNAS L ,et al. Session-based recommendations with recurrent neural networks[C]// Proceedings of 2015 International Conference on Learning Representations.[S.l.:s.n.], 2015.
[41]	LIU Q , WU S , WANG L . Multi-behavioral sequential prediction with recurrent logbilinear model[J]. IEEE Transactions on Knowledge and Data Engineering, 2017,29(6): 1254-1267.
[42]	WU C H , WANG J W , LIU J T ,et al. Recurrent neural network based recommendation for time heterogeneous feedback[J]. Knowledge-Based Systems, 2016,109: 90-103.
[43]	LI Y , LIU T , JIANG J ,et al. Hashtag recommendation with topical attentionbased LSTM[C]// Proceedings of the 26th International Conference on Computational Linguistics. Cambridge:The MIT Press, 2016: 943-952.
[44]	MCMAHAN B , MOORE E , RAMAGE D ,et al. Communication-efficient learning of deep networks from decentralized data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics.[S.l.:s.n.], 2017: 1273-1282.
[45]	KONE?NY J , MCMAHAN B , RAMAGE D . Federated optimization:Distributed optimization beyond the datacenter[J]. arXiv preprint, 2015,arXiv:151103575.
[46]	KAIROUZ E B P , MCMAHAN H B . Advances and open problems in federated learning[J]. Foundations and Trends in Machine Learning, 2021,14(1).
[47]	LI Q B , WEN Z Y , WU Z M ,et al. Federated learning systems:vision,hype and reality for data privacy and protection[J]. arXiv preprint, 2019,arXiv:190709693.
[48]	HARD A , RAO K , MATHEWS R ,et al. Federated learning for mobile keyboard prediction[J]. arXiv preprint, 2018,arXiv:181103604.
[49]	YANG Q , LIU Y , CHEN T J ,et al. Federated machine learning[J]. ACM Transactions on Intelligent Systems and Technology, 2019,10(2): 1-19.
[50]	PAN S J , YANG Q . A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010,22(10): 1345-1359.
[51]	吴建汉, 司世景, 王健宗 ,等. 联邦学习攻击与防御综述[J]. 大数据, 2022:2022038.
	WU J H , SI S J , WANG J Z ,et al. Threats and defenses of federated learning:a survey[J]. Big Data Research, 2022:2022038.
[52]	周俊, 董晓蕾, 曹珍富 . 推荐系统的隐私保护研究进展[J]. 计算机研究与发展, 2019,56(10): 2033-2048.
	ZHOU J , DONG X L , CAO Z F . Research advances on privacy preserving in recommender systems[J]. Journal of Computer Research and Development, 2019,56(10): 2033-2048.
[53]	KIM S , KIM J , KOO D ,et al. Efficient privacy-preserving matrix factorization via fully homomorphic encryption:extended abstract[C]// Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. New York:ACM Press, 2016: 617-628.
[54]	AGRAWAL R , SRIKANT R . Privacypreserving data mining[C]// Proceedings of 2000 ACM SIGMOD International Conference on Management of Data. New York:ACM Press, 2000: 439-50.
[55]	POLAT H , DU W L . Privacy-preserving collaborative filtering using randomized perturbation techniques[C]// Proceedings of 3rd IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2003: 625-628.
[56]	HERLOCKER J , KONSTAN J , BORCHERS A ,et al. An algorithmic framework for performing collaborative filtering[C]// Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 1999: 230-237.
[57]	CHEN K K , LIU L . Privacy preserving data classification with rotation perturbation[C]// Proceedings of 5th IEEE International Conference on Data Mining. Piscataway:IEEE Press, 2005: 589-592.
[58]	DWORK C . A firm foundation for private data analysis[J]. Communications of the ACM, 2011,54(1): 86-95.
[59]	DWORK C , ROTH A . The algorithmic foundations of differential privacy[J]. Foundations and Trends in Theoretical Computer Science, 2013,9(3/4): 211-407.
[60]	MCSHERRY F , MIRONOV I . Differentially private recommender systems:building privacy into the Netflix prize contenders[C]// Proceedings of the 15th ACM SIGKDD International Conference on Knowledge discovery and data mining. New York:ACM Press, 2009: 627-636.
[61]	ZHU T Q , REN Y L , ZHOU W L ,et al. An effective privacy preserving algorithm for neighborhood-based collaborative filtering[J]. Future Generation Computer Systems, 2014,36: 142-155.
[62]	BERLIOZ A , FRIEDMAN A , KAAFAR M A ,et al. Applying differential privacy to matrix factorization[C]// Proceedings of the 9th ACM Conference on Recommender Systems. New York:ACM Press, 2015: 107-114.
[63]	OREKONDY T , OH S J , ZHANG Y ,et al. Gradient-leaks:understanding and controlling deanonymization in federated learning[J]. arXiv preprint, 2018,arXiv:180505838.
[64]	WANG Z B , SONG M K , ZHANG Z F ,et al. Beyond inferring class representatives:user-level privacy leakage from federated learning[C]// Proceedings of 2019 IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2019: 2512-2520.
[65]	MELIS L , SONG C Z , DE CRISTOFARO E ,et al. Exploiting unintended feature leakage in collaborative learning[C]// Proceedings of 2019 IEEE Symposium on Security and Privacy. Piscataway:IEEE Press, 2019: 691-706.
[66]	王健宗, 孔令炜, 黄章成 ,等. 联邦学习隐私保护研究进展[J]. 大数据, 2021,7(3): 130-149.
	WANG J Z , KONG L W , HUANG Z C ,et al. Research advances on privacy protection of federated learning[J]. Big Data Research, 2021,7(3): 130-149.
[67]	周传鑫, 孙奕, 汪德刚 ,等. 联邦学习研究综述[J]. 网络与信息安全学报, 2021,7(5): 77-92.
	ZHOU C X , SUN Y , WANG D G ,et al. Survey of federated learning research[J]. Chinese Journal of Network and Information Security, 2021,7(5): 77-92.
[68]	TRUEX S , LIU L , CHOW K H ,et al. LDP-Fed:federated learning with local differential privacy[C]// Proceedings of the 3rd ACM International Workshop on Edge Systems,Analytics and Networking. New York:ACM Press, 2020: 61-66.
[69]	LIU R X , CAO Y , YOSHIKAWA M ,et al. FedSel:federated SGD under local differential privacy with top-k dimension selection[C]// Proceedings of the International Conference on Database Systems for Advanced Applications. Heidelberg:Springer, 2020: 485-501.
[70]	TRIASTCYN A , FALTINGS B . Federated learning with Bayesian differential privacy[C]// Proceedings of 2019 IEEE International Conference on Big Data. Piscataway:IEEE Press, 2019: 2587-2596.
[71]	BONAWITZ K , IVANOV V , KREUTER B ,et al. Practical secure aggregation for privacy-preserving machine learning[C]// Proceedings of 2017 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2017: 1175-1191.
[72]	LI T , SONG L Q , FRAGOULI C . Federated recommendation system via differential privacy[C]// Proceedings of 2020 IEEE International Symposium on Information Theory. Piscataway:IEEE Press, 2020: 2592-2597.
[73]	LIU S C , XU S Y , YU W H ,et al. FedCT:federated collaborative transfer for recommendation[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2021: 716-725.
[74]	王健宗, 肖京, 朱星华 ,等. 联邦推荐系统的协同过滤冷启动解决方法[J]. 智能系统学报, 2021,16(1): 178-185.
	WANG J Z , XIAO J , ZHU X H ,et al. Cold starts in collaborative filtering for federated recommender systems[J]. CAAI Transactions on Intelligent Systems, 2021,16(1): 178-185.
[75]	WANG L , WANG Y H , BAI Y ,et al. POI recommendation with federated learning and privacy preserving in cross domain recommendation[C]// Proceedings of 2021 IEEE Conference on Computer Communications Workshops. Piscataway:IEEE Press, 2021: 1-6.
[76]	ZONG L L , XIE Q J , ZHOU J H ,et al. FedCMR:federated cross-modal retrieval[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2021: 1672-1676.
[77]	MA J , ZHANG Q C , LOU J ,et al. Communication efficient federated generalized tensor factorization for collaborative health data analytics[C]// Proceedings of the International WorldWide Web Conference. New York:ACM Press, 2021: 171-182.
[78]	CAO L B . Non-IID recommender systems:a review and framework of recommendation paradigm shifting[J]. Engineering, 2016,2(2): 212-224.
[79]	WANG C , CAO L B , WANG M C ,et al. Coupled nominal similarity in unsupervised learning[C]// Proceedings of the 20th ACM international conference on Information and knowledge management. New York:ACM Press, 2011: 973-978.
[80]	WANG C , DONG X J , ZHOU F ,et al. Coupled attribute similarity learning on categorical data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015,26(4): 781-797.
[81]	王健宗, 孔令炜, 黄章成 ,等. 联邦学习算法综述[J]. 大数据, 2020,6(6): 64-82.
	WANG J Z , KONG L W , HUANG Z C ,et al. Research review of federated learning algorithms[J]. Big Data Research, 2020,6(6): 64-82.
[82]	WU Q , HE K , CHEN X . Personalized federated learning for intelligent IoT applications:a cloud-edge based framework[J]. IEEE Computer Graphics and Applications, 2020,1: 35-44.
[83]	YANG C X , WANG Q P , XU M W ,et al. Characterizing impacts of heterogeneity in federated learning upon large-scale smartphone data[C]// Proceedings of 2021 International World-Wide Web Conference.. New York:ACM Press, 2021: 935-946.
[84]	LIU L M , ZHANG J , SONG S H ,et al. Clientedge-cloud hierarchical federated learning[C]// Proceedings of 2020 IEEE International Conference on Communications. Piscataway:IEEE Press, 2020: 1-6.
[85]	XIE C , KOYEJO S , GUPTA I . Asynchronous federated optimization[J]. arXiv preprint,2019,arXiv:1903.03934.
[86]	LI X , HUANG K X , YANG W H ,et al. On the convergence of FedAvg on NonIID data[C]// Proceedings of the 7th International Conference on Learning Representations.[S.l.:s.n.], 2019.
[87]	ZHAO Y , LI M , LAI L Z ,et al. Federated learning with Non-IID data[J]. arXiv preprint, 2018,arXiv:180600582.
[88]	KULKARNI V , KULKARNI M , PANT A . Survey of personalization techniques for federated learning[C]// Proceedings of 2020 4th World Conference on Smart Trends in Systems,Security and Sustainability. Piscataway:IEEE Press, 2020: 794-797.
[89]	WU J Z , LIU Q , HUANG Z Y ,et al. Hierarchical personalized federated learning for user modeling[C]// Proceedings of 2021 Web Conference. New York:ACM Press, 2021: 957-968.
[90]	ROTHCHILD D , PANDA A , ULLAH E ,et al. FetchSGD:communication-efficient federated learning with sketching[C]// Proceedings of the 37th International Conference on Machine Learning. New York:ACM Press, 2020.
[91]	REISIZADEH A , MOKHTARI A , HASSANI H ,et al. FedPAQ:a communication-efficient federated learning method with periodic averaging and quantization[C]// Proceedings of 2020 International Conference on Artificial Intelligence and Statistics.[S.l.:s.n.], 2020: 2021-2031.
[92]	KHAN F K , FLANAGAN A , TAN K E ,et al. A payload optimization method for federated recommender systems[C]// Proceedings of the 15th ACM Conference on Recommender Systems. New York:ACM Press, 2021: 432-442.
[93]	MALINOVSKY G , KOVALEV D , GASANOV E ,et al. From local SGD to local fixed point methods for federated learning[C]// Proceedings of the 37th International Conference on Machine Learning. New York:ACM Press, 2020: 6692-6701.
[94]	KONE?NY J , MCMAHAN H B , YU F X ,et al. Federated learning:strategies for improving communication efficiency[C]// Proceedings of the NIPS Workshop on Private Multi-Party Machine Learning. Cambridge:The MIT Press, 2016.
[95]	MANSOUR Y , MOHRI M , RO J ,et al. Three approaches for personalization with applications to federated learning[J]. arXiv preprint,2020,arXiv:2002.10619.
[96]	SMITH V , CHIANG C K , SANJABI M ,et al. Federated multi-task learning[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems.Red Hook:Curran Associates Inc. , 2017: 4427-4437.
[97]	LINDEN G , SMITH B , YORK J.Amazon . com recommendations:item-to-item collaborative filtering[J]. IEEE Internet Computing, 2003,7(1): 76-80.
[98]	LI D L , WANG J P . FedMD:heterogenous federated learning via model distillation[J]. arXiv preprint,2019,arXiv:1910.03581.
[99]	ARIVAZHAGAN M G , AGGARWAL V , SINGH A K ,et al. Federated learning with personalization layers[J]. arXiv preprint,2019,arXiv:1912.00818.
[100]	HANZELY F , RICHTáRIK P , . Federated learning of a mixture of global and local models[J]. arXiv preprint,2020,arXiv:2002.05516.
[101]	WANG J Y , JOSHI G . Cooperative SGD:a unified framework for the design and analysis of communication-efficient SGD algorithms[C]// Proceedings of the ICML Workshop on Coding Theory for Machine Learning.[S.l.:s.n.], 2019.
[102]	LI X , YANG W H , WANG S S ,et al. Communication efficient decentralized training with multiple local updates[J]. arXiv preprint,2019,arXiv:1910.09126.
[103]	LIANG P P , LIU T , ZIYIN L ,et al. Think locally,act globally:federated learning with local and global representations[J]. arXiv preprint, 2020,arXiv:200101523.
[104]	LIU Y , KANG Y , ZHANG X W ,et al. A communication efficient vertical federated learning framework[J]. arXiv preprint,2019,arXiv:1912.11187.
[105]	马嘉华, 孙兴华, 夏文超 ,等. 基于标签量信息的联邦学习节点选择算法[J]. 物联网学报, 2021,5(4): 46-53.
	MA J H , SUN X H , XIA W C ,et al. Node selection based on label quantity information in federated learning[J]. Chinese Journal on Internet of Things, 2021,5(4): 46-53.
[106]	AMMAD-UD-DIN M , IVANNIKOVA E , KHAN S A ,et al. Federated collaborative filtering for privacy-preserving personalized recommendation system[J]. arXiv preprint, 2019,arXiv:190109888.
[107]	CHAI D , WANG L Y , CHEN K ,et al. Secure federated matrix factorization[J]. IEEE Intelligent Systems, 2021,36(5): 11-20.
[108]	MINTO L , HALLER M , LIVSHITS B ,et al. Stronger privacy for federated collaborative filtering with implicit feedback[C]// Proceedings of the 15th ACM Conference on Recommender Systems. New York:ACM Press, 2021: 342-350.
[109]	LIN G Y , LIANG F , PAN W K ,et al. FedRec:federated recommendation with explicit feedback[J]. IEEE Intelligent Systems, 2021,36(5): 21-30.
[110]	LIANG F , PAN W , MING Z . FedRec++:lossless federated recommendation with explicit feedback[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2021: 4224-4231.
[111]	QI T , WU F Z , WU C H ,et al. Privacypreserving news recommendation model learning[C]// Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing:Findings.[S.l.]: ACL Press, 2020: 1423-32.
[112]	OKURA S , TAGAMI Y , ONO S ,et al. Embedding-based news recommendation for millions of users[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2017: 1933-1942.
[113]	HUANG M K , LI H , BAI B ,et al. A federated multi-view deep learning framework for privacy-preserving recommendations[J]. arXiv preprint,2020,arXiv:2008.10808.
[114]	HUANG P S , HE X D , GAO J F ,et al. Learning deep structured semantic models for web search using clickthrough data[C]// Proceedings of the 22nd ACM International Conference on Information ＆Knowledge Management. New York:ACM Press, 2013: 2333-2338.
[115]	LIN Y J , REN P J , CHEN Z M ,et al. Meta matrix factorization for federated rating predictions[C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2020: 981-990.
[116]	MUHAMMAD K , WANG Q Q , O’REILLYMORGAN D ,et al. FedFast:going beyond average for faster training of federated recommender systems[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆Data Mining. New York:ACM Press, 2020: 1234-1242.
[117]	KHARITONOV E , . Federated online learning to rank with evolution strategies[C]// Proceedings of the 12th ACM International Conference on Web Search and Data Mining. New York:ACM Press, 2019: 249-257.
[118]	TRIENES J , CANO A T , HIEMSTRA D . Recommending users:whom to follow on federated social networks[J]. arXiv preprint, 2018,arXiv:181109292.
[119]	TAN B , LIU B , ZHENG V ,et al. A federated recommender system for online services[C]// Proceedings of the 14th ACM Conference on Recommender Systems. New York:ACM Press, 2020: 579-581.
[120]	RIBERO M , HENDERSON J , WILLIAMSON S ,et al. Federating recommendations using differentially private prototypes[J]. arXiv preprint, 2020,arXiv:200300602.
[121]	HU H S , DOBBIE G , SALCIC Z ,et al. A locality sensitive hashing based approach for federated recommender system[C]// Proceedings of 2020 20th IEEE/ACM International Symposium on Cluster,Cloud and Internet Computing. Piscataway:IEEE Press, 2020: 836-842.
[122]	BLANCHARD P , MHAMDI E M E , GUERRAOUI R ,et al. Machine learning with adversaries:Byzantine tolerant gradient descent[C]// Proceedings of the 31st Annual Conference on Neural Information Processing Systems.Red Hook:Curran Associates Inc. , 2017: 118-128.
[123]	MHAMDI E M E , GUERRAOUI R , ROUAULT S E B . The hidden vulnerability of distributed learning in Byzantium[C]// Proceedings of the 34th International Conference on Machine Learning. New York:ACM Press, 2018.
[124]	WANG H , YUROCHKIN M , SUN Y ,et al. Federated learning with matched averaging[J]. arXiv preprint,2020,arXiv:2002.06440.
[125]	YAN B J , LIU B Y , WANG L J ,et al. FedCM:a real-time contribution measurement method for participants in federated learning[C]// Proceedings of 2021 International Joint Conference on Neural Networks. Piscataway:IEEE Press, 2021: 1-8.
[126]	WANG G , DANG C X , ZHOU Z Y . Measure contribution of participants in federated learning[C]// Proceedings of 2019 IEEE International Conference on Big Data. Piscataway:IEEE Press, 2019: 2597-2604.
[127]	ZHANG J F , LI C , ROBLES-KELLY A , ,et al. Hierarchically fair federated learning[J]. arXiv preprint,2020,arXiv:2004.10386.

贡献与挑战	方面	方法	参考文献
联邦推荐系统的贡献	隐私保护	加密方法	[53,71]
		数据扰动	[63,65,68-72]
	跨领域推荐	跨领域用户信息	[73,75]
		跨模态用户数据	[76-77]
落地过程中的挑战	数据异质性	设备异质性	[83-85]
		统计异质性	[1,86-87]
		模型异质性	[88-89,95-100]
	通信成本	通信优化方法	[90,94,101-104]

模型	介绍	解决问题	局限性
FCF^[106]	针对隐反馈数据首先提出联邦协同过滤算法	在保护隐私数据的前提下共同训练协同过滤推荐模型	需要所有客户端参与；存在冷启动问题；存在梯度泄露风险
SFMF^[107]	加入同态加密进行安全矩阵分解	防止梯度泄露	在隐私保护力度与算力开销之间难以两全
Stronger FCF^[108]	使用本地差分隐私与代理服务器保护物品更新梯度矩阵	防止第三方获取物品更新梯度矩阵实施重构攻击	仅在隐反馈推荐情景下证明了有效性
FedRec ^[109]、FedRec++^[110]	适用于显反馈评分数据的联邦协同过滤，后者增加了消除噪声的机制	对基于显反馈数据的推荐系统进行了联邦化实践	与现代推荐算法的结合不多
FedNewsRec^[111]	结合深度网络学习新闻表征与用户历史，产生新闻推荐	解决冷启动问题；不需要所有用户参与	非通用模型
FL-MV-DSSM^[113]	多视图深度结构化语义模型	解决冷启动问题；多视图	带来新的安全挑战
MetaMF^[115]	引入元学习矩阵分解训练个性化模型	快速适应新用户	仅适用于评分预测任务
FedFast^[116]	在客户端选择与更新信息聚合方面提出改善方法	加快模型收敛，减少计算量	带来额外的通信开销

联邦推荐系统综述

Survey on federated recommendation systems

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 127

相关文章 15

Metrics

推荐阅读 0

[1]	邓钇敏, 张旭龙, 司世景, 王健宗, 肖京. 虚拟人形象合成技术综述[J]. 大数据, 2023, 9(3): 114-139.
[2]	张传尧, 司世景, 王健宗, 肖京. 联邦元学习综述[J]. 大数据, 2023, 9(2): 122-146.
[3]	贺亚运, 彭俊清, 王健宗, 肖京. 节奏舞者：基于关键动作转换图和有条件姿态插值网络的3D舞蹈生成方法研究[J]. 大数据, 2023, 9(1): 23-37.
[4]	崔雨萌, 王靖亚, 闫尚义, 陶知众. 基于深度学习的警情记录关键信息自动抽取[J]. 大数据, 2022, 8(6): 127-142.
[5]	吴建汉, 司世景, 王健宗, 肖京. 联邦学习攻击与防御综述[J]. 大数据, 2022, 8(5): 12-32.
[6]	张宇奇, 黄晓雯, 桑基韬. 知识增强策略引导的交互式强化推荐系统[J]. 大数据, 2022, 8(5): 88-105.
[7]	尹虹舒, 周旭华, 周文君. 纵向联邦线性模型在线推理过程中成员推断攻击的隐私保护研究[J]. 大数据, 2022, 8(5): 45-54.
[8]	阮雯强, 徐铭辛, 涂新宇, 宋鲁杉, 韩伟力. 数据租赁——数据流通的新方式[J]. 大数据, 2022, 8(5): 3-11.
[9]	李懿, 王劲松, 张洪玮. 基于区块链与函数加密的隐私数据安全共享模型研究[J]. 大数据, 2022, 8(5): 33-44.
[10]	张燕, 杨一帆, 伊人, 罗圣美, 唐剑飞, 夏正勋. 隐私计算场景下数据质量治理探索与实践[J]. 大数据, 2022, 8(5): 55-73.
[11]	许小颖, 陈熙, 陈源, 谢永靖. 区块链在个性化推荐系统中的应用研究综述[J]. 大数据, 2022, 8(3): 87-102.
[12]	王杰, 张松岩, 梁吉业. 融合一致性正则与流形正则的半监督深度学习算法[J]. 大数据, 2022, 8(3): 103-114.
[13]	徐康庭, 宋威. 结合语言知识和深度学习的中文文本情感分析方法[J]. 大数据, 2022, 8(3): 115-127.
[14]	赵智韬, 赵理君, 张正, 唐娉. 基于容器云技术的典型遥感智能解译算法集成[J]. 大数据, 2022, 8(2): 58-74.
[15]	张凯, 车漾. 基于分布式缓存加速容器化深度学习的优化方法[J]. 大数据, 2021, 7(5): 150-163.