基于艾宾浩斯遗忘曲线和注意力机制的推荐算法

doi:10.11959/j.issn.1000-0801.2022266

电信科学 ›› 2022, Vol. 38 ›› Issue (10): 89-97.doi: 10.11959/j.issn.1000-0801.2022266

基于艾宾浩斯遗忘曲线和注意力机制的推荐算法

金楠¹, 王瑞琴¹^,², 陆悦聪¹

¹ 湖州师范学院信息工程学院，浙江湖州 313000
² 浙江省现代农业资源智慧管理与应用研究重点实验室，浙江湖州 313000

修回日期:2022-09-28 出版日期:2022-10-20 发布日期:2022-10-01
作者简介:金楠（1996- ），男，湖州师范学院信息工程学院硕士生，主要研究方向为深度学习、个性化推荐
王瑞琴（1979- ），女，湖州师范学院信息工程学院教授、硕士生导师，主要研究方向为自然语言理解、数据挖掘、个性化推荐
陆悦聪（1996- ），男，湖州师范学院信息工程学院硕士生，主要研究方向为机器学习与数据挖掘
基金资助:
浙江省自然科学基金资助项目(LY20F020006)

Ebbinghaus forgetting curve and attention mechanism based recommendation algorithm

Nan JIN¹, Ruiqin WANG¹^,², Yuecong LU¹

¹ School of Information Engineering, Huzhou University, Huzhou 313000, China
² Zhejiang Province Key Laboratory of Smart Management ＆Application of Modern Agricultural Resources, Huzhou 313000, China

Revised:2022-09-28 Online:2022-10-20 Published:2022-10-01
Supported by:
The Natural Science Foundation of Zhejiang Province(LY20F020006)

摘要/Abstract

摘要：

传统基于注意力机制的推荐算法只利用位置嵌入对用户行为序列进行建模，忽略了具体的时间戳信息，导致推荐性能不佳和模型训练过拟合等问题。提出基于时间注意力的多任务矩阵分解推荐模型，利用注意力机制提取邻域信息对用户和物品进行嵌入编码，借助艾宾浩斯遗忘曲线描述用户兴趣随时间的变化特性，在模型训练过程中引入经验回放的强化学习策略模拟人类的记忆复习过程。真实数据集上的实验结果表明，该模型比现有推荐模型具有更好的推荐性能。

关键词: 艾宾浩斯遗忘曲线, 注意力机制, 强化学习, 经验回放

Abstract:

Traditional attention-based recommendation algorithms only use position embeddings to model user behavior sequences, however, ignore specific timestamp information, resulting in poor recommendation performance and overfitting of model training.The multi-task matrix factorization recommendation model based on time attention was proposed, which used the attention mechanism to extract the neighborhood information for the user and item embedding, and used the Ebbinghaus forgetting curve to describe the changing characteristics of user interests over time.The model training process introduced a reinforcement learning strategy of experience replay to simulate the human memory review process.Experimental results on real datasets show that the proposed model has better recommendation performance than existing recommendation models.

Key words: Ebbinghaus forgetting curve, attention mechanism, reinforcement learning, experience replay

中图分类号:

TP391

金楠, 王瑞琴, 陆悦聪. 基于艾宾浩斯遗忘曲线和注意力机制的推荐算法[J]. 电信科学, 2022, 38(10): 89-97.

Nan JIN, Ruiqin WANG, Yuecong LU. Ebbinghaus forgetting curve and attention mechanism based recommendation algorithm[J]. Telecommunications Science, 2022, 38(10): 89-97.

图/表 11

图1

图2

图3

图4

表1

表2

图5

图6

图7

图8

表3

参考文献 22

[1]	黄立威, 江碧涛, 吕守业 ,等. 基于深度学习的推荐系统研究综述[J]. 计算机学报, 2018,41(7): 1619-1647.
	HUANG L W , JIANG B T , LYU S Y ,et al. Survey on deep learning based recommender systems[J]. Chinese Journal of Computers, 2018,41(7): 1619-1647.
[2]	纪佳琪, 姜学东 . 深度协同过滤推荐模型研究[J]. 计算机应用与软件, 2020,37(7): 240-245.
	JI J Q , JIANG X D . Deep collaborative filtering recommender model[J]. Computer Applications and Software, 2020,37(7): 240-245.
[3]	HE X N , LIAO L Z , ZHANG H W ,et al. Neural collaborative filtering[C]// Proceedings of the 26th International Conference on World Wide Web.[S.l.:s.n.], 2017: 173-182.
[4]	VEIT A , WILBER M J , BELONGIE S . Residual networks behave like ensembles of relatively shallow networks[J]. Advances in Neural Information Processing Systems, 2016,29: 550-558.
[5]	DENG Z H , HUANG L , WANG C D ,et al. DeepCF:a unified framework of representation learning and matching function learning in recommender system[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019,33: 61-68.
[6]	SHEN T , ZHOU T Y , LONG G D ,et al. DiSAN:directional self-attention network for RNN/CNN-free language understanding[C]// Proceedings of the AAAI Conference on Artificial Intelligence, 2018,32(1).
[7]	ZHANG W F , YU J , HU H ,et al. Multimodal feature fusion by relational reasoning and attention for visual question answering[J]. Information Fusion, 2020,55: 116-126.
[8]	WANG X , HE X N , WANG M ,et al. Neural graph collaborative filtering[C]// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.[S.l.:s.n], 2019: 165-174.
[9]	WU S , TANG Y Y , ZHU Y Q ,et al. Session-based recommendation with graph neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019,33(1): 346-353.
[10]	LU Y F , XIE R B , SHI C ,et al. Social influence attentive neural network for friend-enhanced recommendation[C]// Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases.[S.l.:s.n.], 2020: 3-18.
[11]	WANG Z , LIN G Y , TAN H B ,et al. CKAN:collaborative knowledge-aware attentive network for recommender systems[C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2020: 219-228.
[12]	WANG Z Y , WEI W , CONG G ,et al. Global context enhanced graph neural networks for session-based recommendation[C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM Press, 2020: 169-178.
[13]	XIA X , YIN H Z , YU J L ,et al. Self-supervised hypergraph convolutional networks for session-based recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(5): 4503-4511.
[14]	ZHANG Y , YANG Q . An overview of multi-task learning[J]. National Science Review, 2017,5(1): 30-43.
[15]	LU Y C , DONG R H , SMYTH B . Why I like it:multi-task learning for recommendation and explanation[C]// Proceedings of the 12th ACM Conference on Recommender Systems. New York:ACM Press, 2018: 4-12.
[16]	XIAO Y , LI C D , LIU V . DFM-GCN:a multi-task learning recommendation based on a deep graph neural network[J]. Mathematics, 2022,10(5): 721.
[17]	WANG Y Q , DONG L Y , LI Y L ,et al. Multitask feature learning approach for knowledge graph enhanced recommendations with RippleNet[J]. PLoS One, 2021,16(5): e0251162.
[18]	LIU W W , ZHANG Y , WANG J L ,et al. Item relationship graph neural networks for E-commerce[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022,33(9): 4785-4799.
[19]	LINDEN G , SMITH B , YORK J.Amazon . com recommendations:item-to-item collaborative filtering[J]. IEEE Internet Computing, 2003,7(1): 76-80.
[20]	KOREN Y , BELL R , VOLINSKY C . Matrix factorization techniques for recommender systems[J]. Computer, 2009,42(8): 30-37.
[21]	ZHENG S , DING C , NIE F P . Regularized singular value decomposition and application to recommender system[J]. arXiv preprint,2018,arXiv:1804.05090.
[22]	MOLCHANOV P , TYREE S , KARRAS T ,et al. Pruning convolutional neural networks for resource efficient inference[J]. arXiv preprint,2016,arXiv:1611.06440.

数据集	FilmTrust	MovieLens 100K	MovieLens 1M	MovieLens Last
用户数/位	1 508	943	6 040	610
电影数/部	2 071	1 682	3 706	9 742
评分数/个	35 497	100 000	1 000 209	100 836
评分范围	[0.5,4]	[1,5]	[1,5]	[1,5]
评分密度	1.14%	6.30%	4.47%	1.70%

模型	FilmTrust		MovieLens 100K		MovieLens 1M		MovieLens Last
模型	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
ItemCF	0.657 3	0.882 4	0.749 1	0.969 7	0.731 3	0.955 6	0.818 1	1.041 3
PMF	0.640 5	0.823 1	0.753 2	0.933 5	0.735 6	0.897 1	0.685 4	0.894 2
MLP	0.648 2	0.824 2	0.725 0	0.929 6	0.692 8	0.880 0	0.667 2	0.871 9
SVD	0.634 6	0.822 8	0.746 7	0.922 3	0.721 8	0.907 5	0.665 2	0.852 3
GCN	0.631 7	0.815 7	0.700 3	0.896 2	0.689 2	0.862 2	0.651 6	0.848 0
NeuMF	0.632 6	0.810 6	0.712 3	0.906 2	0.687 6	0.859 3	0.650 2	0.849 8
DeepCF	0.632 0	0.817 4	0.698 0	0.895 0	0.668 8	0.852 4	0.647 6	0.842 5
TAMMF	0.622 0	0.803 0	0.697 2	0.893 3	0.667 1	0.846 9	0.644 7	0.838 8
TAMMF+	0.614 3	0.798 8	0.694 9	0.888 9	0.662 8	0.845 6	0.631 8	0.832 2

数据集	性能	size=32	size=64	size=128	size=256
FilmTrust	RMSE	0.812 6	0.807 6	0.798 8	0.802 6
	MAE	0.623 3	0.619 3	0.614 3	0.617 3
ML-100K	RMSE	0.906 1	0.896 1	0.888 9	0.894 9
	MAE	0.708 1	0.698 9	0.694 9	0.698 2
ML-Last	RMSE	0.846 1	0.836 4	0.832 2	0.834 4
	MAE	0.656 3	0.636 1	0.631 8	0.639 1
ML-1M	RMSE	0.853 4	0.843 4	0.845 6	0.848 4
	MAE	0.681 0	0.669 0	0.662 8	0.666 0

基于艾宾浩斯遗忘曲线和注意力机制的推荐算法

Ebbinghaus forgetting curve and attention mechanism based recommendation algorithm

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 22

相关文章 15

Metrics

推荐阅读 0

[1]	胡珈玮, 刘晓谦, 唐昕柯, 董宇涵. 基于DQN的UUV辅助水下无线光通信轨迹规划系统[J]. 电信科学, 2023, 39(5): 42-47.
[2]	张永, 刘纪奎, 柯文龙. 基于并行可分离卷积和标签平滑正则化的脑电情感识别[J]. 电信科学, 2023, 39(5): 116-128.
[3]	王雪荣, 唐政治, 李银川, 齐美玉, 朱建波, 张亮. 基于优化决策树的时延敏感流智能感知调度[J]. 电信科学, 2023, 39(4): 120-132.
[4]	廖熙雯, 冷甦鹏, 明昱君, 李天扬. 基于数字孪生的城市交通流智能预测与导引策略[J]. 电信科学, 2023, 39(3): 70-79.
[5]	戴剑锋, 陈星妤, 董黎刚, 蒋献. 融合混合嵌入与关系标签嵌入的三元组联合抽取方法[J]. 电信科学, 2023, 39(2): 132-144.
[6]	康宇, 刘雅琼, 赵彤雨, 寿国础. AI算法在车联网通信与计算中的应用综述[J]. 电信科学, 2023, 39(1): 1-19.
[7]	邓丹昊, 王朝炜, 江帆, 王卫东. 无人机辅助无蜂窝大规模MIMO中的空地协同调度[J]. 电信科学, 2022, 38(8): 37-44.
[8]	李昕, 孙君. 基于价值差异学习的多小区mMTC接入算法[J]. 电信科学, 2022, 38(6): 82-90.
[9]	宋勇, 严志伟, 秦玉坤, 赵东明, 叶晓舟, 柴园园, 欧阳晔. 基于矩阵分解和注意力多任务学习的客服投诉工单分类[J]. 电信科学, 2022, 38(2): 103-110.
[10]	孙金杨, 刘柏嵩, 任豪, 钱江波. TAGAN：一种融合细粒度语义特征的学术论文对抗推荐算法[J]. 电信科学, 2021, 37(8): 57-65.
[11]	章坚武, 王路鑫, 孙玲芬, 章谦骅, 单杭冠. 人工智能在5G系统中的应用综述[J]. 电信科学, 2021, 37(5): 14-31.
[12]	伍仲丽, 曹园园, 黄文睿, 戴彬, 莫益军. 面向确定性网络的按需智能路由技术[J]. 电信科学, 2021, 37(11): 11-16.
[13]	桂飞,程阳,李丹,洪思虹. 互联网智能路由架构及算法[J]. 电信科学, 2020, 36(10): 12-20.
[14]	何明捷,张杰,山世光. 神经结构搜索进展概述[J]. 电信科学, 2019, 35(5): 43-50.
[15]	王一鸣,陈恳,萨阿卜杜萨拉木·艾海提拉木. 基于SDBN和BLSTM注意力融合的端到端视听双模态语音识别[J]. 电信科学, 2019, 35(12): 79-89.