机器人持续学习进展与展望

doi:10.11959/j.issn.2096-6652.202235

智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (3): 308-323.doi: 10.11959/j.issn.2096-6652.202235

机器人持续学习进展与展望

赵超¹^,², 许杰¹^,², 陈星宇¹^,², 梅魁志¹^,², 兰旭光¹^,²

¹ 西安交通大学人工智能与机器人研究所，陕西西安 710049
² 西安交通大学人工智能学院，陕西西安 710049

修回日期:2022-06-30 出版日期:2022-09-15 发布日期:2022-09-01
作者简介:赵超（1996- ），男，西安交通大学人工智能与机器人研究所博士生，主要研究方向为机器人持续学习
许杰（1996- ），男，西安交通大学人工智能与机器人研究所博士生，主要研究方向为人机交互、人体运动预测
陈星宇（1990- ），男，博士，西安交通大学人工智能与机器人研究所助理教授，主要研究方向为计算机视觉、机器人环境感知、人-机器人共融协作
梅魁志（1978- ），男，博士，西安交通大学人工智能与机器人研究所教授、博士生导师，主要研究方向为机器人场景感知与定位、深度学习/智能系统的软硬件协同设计
兰旭光（1976- ），男，博士，西安交通大学人工智能与机器人研究所教授、博士生导师，主要研究方向计算机视觉、统计学习与模式识别、深度强化学习与机器人学习、人-机器人共融协作
基金资助:
国家自然科学基金资助项目(62125305);国家自然科学基金资助项目(62076193);国家自然科学基金资助项目(62088102);科技创新2030—“新一代人工智能”重大项目(2021ZD0112700);中央高校基础研究基金资助项目(xtr072022001)

A review of continual learning for robotics

Chao ZHAO¹^,², Jie XU¹^,², Xingyu CHEN¹^,², Kuizhi MEI¹^,², Xuguang LAN¹^,²

¹ Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, China
² College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an 710049, China

Revised:2022-06-30 Online:2022-09-15 Published:2022-09-01
Supported by:
The National Natural Science Foundation of China(62125305);The National Natural Science Foundation of China(62076193);The National Natural Science Foundation of China(62088102);The National Key Research and Development Program of China(2021ZD0112700);The Fundamental Research Funds for the Central Universities(xtr072022001)

摘要/Abstract

摘要：

当前机器人技术面临的较大限制之一是难以适应不断变化的任务，当机器人面对新环境或者学习新任务时，会不可避免地遗忘旧环境或旧任务的经验。为了总结机器人持续学习的研究和发展现状，首先介绍了持续学习的框架和评价基准，然后阐述了持续学习在机器人任务中的必要性和面临的挑战，并对持续学习的发展现状进行了回顾，最后展望了机器人持续学习的发展前景，提出了一些有价值的研究问题。

关键词: 持续学习, 终身学习, 机器人

Abstract:

One of the limitations of robotics is that it is difficult for robots to adapt to fickle tasks.A robot will inevitably forget the knowledge from old environments or tasks when facing new environments or tasks.In order to summarize research in continual learning for robotics, firstly, the framework and evaluation protocols in continual learning were introduced.And then necessity and challenge of continual learning in the robotics were expounded.The research for continual learning was also summarized.Finally, the prospect of continual learning was predicted and some valuable research directions were put forward.

Key words: continual learning, lifelong learning, robotics

中图分类号:

TP242.6

赵超, 许杰, 陈星宇, 等. 机器人持续学习进展与展望[J]. 智能科学与技术学报, 2022, 4(3): 308-323.

Chao ZHAO, Jie XU, Xingyu CHEN, et al. A review of continual learning for robotics[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 308-323.

图/表 11

图1

表1

本文使用的主要符号的含义"

符号	含义
T_i	第i个任务
D_i	第i个任务的数据集
T	任务数量
R_{i ,j}	模型学习完任务T_i后在任务T_j上的测试准确率
$\bar{b_{i}}$	随机初始化的任务T_i的测试准确率

表1

图2

图3

图4

图5

图6

图7

图8

图9

图10

参考文献 119

[1]	RING M B . Continual learning in reinforcement environments[M]. Austin: University of Texas, 1994.
[2]	TJOMSLAND J , KALKAN S , GUNES H . Mind your manners! A dataset and a continual learning approach for assessing social appropriateness of robot actions[J]. Frontiers in Robotics and AI, 2022,9:669420.
[3]	GROSSMANN A . Continual learning for mobile robots[D]. Birmingham:University of Birmingham, 2001.
[4]	NEHMZOW U , RIANO L . A proposal for continual learning in robotics[C]// Proceedings of Towards Autonomous Robotic Systems.[S.l.:s.n.], 2009.
[5]	LIU B , XIAO X S , STONE P . A lifelong learning approach to mobile robot navigation[J]. IEEE Robotics and Automation Letters, 2021,6(2): 1090-1096.
[6]	HADSELL R , RAO D , RUSU A A ,et al. Embracing change:continual learning in deep neural networks[J]. Trends in Cognitive Sciences, 2020,24(12): 1028-1040.
[7]	PARISI G I , KEMKER R , PART J L ,et al. Continual lifelong learning with neural networks:a review[J]. Neural Networks, 2019,113: 54-71.
[8]	DE LANGE M , ALJUNDI R , MASANA M ,et al. A continual learning survey:defying forgetting in classification tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022,44(7): 3366-3385.
[9]	VERWIMP E , DE LANGE M , TUYTELAARS T . Rehearsal revealed:the limits and merits of revisiting samples in continual learning[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 9365-9374.
[10]	LESORT T , LOMONACO V , STOIAN A ,et al. Continual learning for robotics:definition,framework,learning strategies,opportunities and challenges[J]. Information Fusion, 2020,58: 52-68.
[11]	LIU Y Y , SCHIELE B , SUN Q R . Adaptive aggregation networks for class-incremental learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 2544-2553.
[12]	ZHAI M Y , CHEN L , HE J W ,et al. Piggyback GAN:efficient lifelong learning for image conditioned generation[M]// Computer vision -ECCV 2020. Cham: Springer International Publishing, 2020: 397-413.
[13]	MITCHELL T , COHEN W , HRUSCHKA E ,et al. Never-ending learning[J]. Communications of the ACM, 2018,61(5): 103-115.
[14]	MCCLOSKEY M , COHEN N J . Catastrophic interference in connectionist networks:the sequential learning problem[J]. Psychology of Learning and Motivation, 1989,24: 109-165.
[15]	TRAORé R , CASELLES-DUPRé H , LESORT T ,et al. Continual reinforcement learning deployed in real-life using policy distillation and Sim2Real transfer[C]// Proceedings of ICML Workshop on Multi-Task and Lifelong Reinforcement Learning.[S.l.:s.n.], 2019.
[16]	ZHANG Y , YANG Q . A survey on multi-task learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2021.
[17]	PAN S J , YANG Q . A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010,22(10): 1345-1359.
[18]	HOI S C H , SAHOO D , LU J ,et al. Online learning:a comprehensive survey[J]. Neurocomputing, 2021,459: 249-289.
[19]	GOODFELLOW I J , MIRZA M , XIAO D ,et al. An empirical investigation of catastrophic forgetting in gradient-based neural networks[C]// Proceedings of the 2nd International Conference on Learning Representations.[S.l.:s.n.], 2014.
[20]	LOPEZ-PAZ D , RANZATO M . Gradient episodic memory for continual learning[J]. arXiv preprint,2017,arXiv:1706.08840.
[21]	REBUFFI S A , KOLESNIKOV A , SPERL G ,et al. iCaRL:incremental classifier and representation learning[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 5533-5542.
[22]	SAENKO K , KULIS B , FRITZ M ,et al. Adapting visual category models to new domains[J]. Lecture Notes in Computer Science, 2010: 213-226.
[23]	LECUN Y , BOTTOU L , BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.
[24]	GANIN Y , LEMPITSKY V . Unsupervised domain adaptation by backpropagation[C]// Proceedings of the 32nd International Conference on Machine Learning.[S.l.:s.n.], 2015: 1180-1189.
[25]	HULL J J . A database for handwritten text recognition research[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994,16(5): 550-554.
[26]	NETZER Y , WANG T . Reading digits in natural images with unsupervised feature learning[J]. Advances in Neural Information Processing Systems, 2011.
[27]	BELLEMARE M G , NADDAF Y , VENESS J ,et al. The arcade learning environment:an evaluation platform for general agents[J]. Journal of Artificial Intelligence Research, 2013,47: 253-279.
[28]	JUNG S , AHN H , CHA S ,et al. Continual learning with node-importance based adaptive group sparse regularization[J]. Advances in Neural Information Processing Systems, 2020,33: 3647-3658.
[29]	RUSU A A , RABINOWITZ N C , DESJARDINS G ,et al. Progressive neural networks[J]. arXiv preprint,2016,arXiv:1606.04671.
[30]	WANG L Y , ZHANG M T , JIA Z F ,et al. AFEC:active forgetting of negative transfer in continual learning[J]. arXiv preprint,2021,arXiv:2110.12187.
[31]	MENDEZ J A , WANG B Y , EATON E . Lifelong policy gradient learning of factored policies for faster training without forgetting[J]. Advances in Neural Information Processing Systems, 2020.
[32]	XU K , VERMA S , FINN C ,et al. Continual learning of control primitives:skill discovery via reset-games[J]. Advances in Neural Information Processing Systems, 2020.
[33]	YANG F , YANG C , LIU H ,et al. Evaluations of the gap between supervised and reinforcement lifelong learning on robotic manipulation tasks[C]// Proceedings of the Conference on Robot Learning.[S.l.:s.n.], 2022: 547-556.
[34]	TODOROV E , EREZ T , TASSA Y . MuJoCo:a physics engine for model-based control[C]// Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2012: 5026-5033.
[35]	BROCKMAN G , CHEUNG V , PETTERSSON L ,et al. OpenAI Gym[J]. arXiv preprint,2016,arXiv:1606.01540.
[36]	JULIAN R , SWANSON B , SUKHATME G S ,et al. Never stop learning:the effectiveness of fine-tuning in robotic reinforcement learning[J]. arXiv preprint,2020,arXiv:2004.10190.
[37]	DíAZ-RODRíGUEZ N , LOMONACO V , FILLIAT D ,et al. Don’t forget,there is more than forgetting:new metrics for continual learning[J]. arXiv preprint,2018,arXiv:1810.13166.
[38]	CHAUDHRY A , DOKANIA P K , AJANTHAN T ,et al. Riemannian walk for incremental learning:understanding forgetting and intransigence[C]// Proceedings of the European Conference on Computer Vision.[S.l.:s.n.], 2018: 532-547.
[39]	YIN H , YANG P , LI P . Mitigating forgetting in online continual learning with neuron calibration[J]. Advances in Neural Information Processing Systems, 2021,34.
[40]	WO?CZYK M , ZAJ?C M , PASCANU R ,et al. Continual world:a robotic benchmark for continual reinforcement learning[J]. Advances in Neural Information Processing Systems, 2021.
[41]	JIANG Y , BHARADWAJ S , WU B ,et al. Temporal-logic-based reward shaping for continuing learning tasks[J]. arXiv preprint,2020,arXiv:2007.01498.
[42]	KEMKER R , MCCLURE M , ABITINO A ,et al. Measuring catastrophic forgetting in neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018,32(1): 3390-3398.
[43]	RIEMER M , CASES I , AJEMIAN R ,et al. Learning to learn without forgetting by maximizing transfer and minimizing interference[C]// Proceedings of the 7th International Conference on Learning Representations.[S.l.:s.n.], 2019.
[44]	HENNING C , CERVERA M , D’ANGELO F , ,et al. Posterior meta-replay for continual learning[J]. Advances in Neural Information Processing Systems, 2021,34.
[45]	DONG J , CONG Y , SUN G ,et al. I3DOL:Incremental 3D object learning without catastrophic forgetting[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2021: 6066-6074.
[46]	ZHOU M , XIAO J , CHANG Y F ,et al. Image de-raining via continual learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 4905-4914.
[47]	CHERAGHIAN A , RAHMAN S , RAMASINGHE S ,et al. Synthesized feature based few-shot class-incremental learning on a mixture of subspaces[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 8641-8650.
[48]	KUKLEVA A , KUEHNE H , SCHIELE B . Generalized and incremental few-shot learning by explicit learning and calibration without forgetting[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 9000-9009.
[49]	LOO N , SWAROOP S , TURNER R E . generalized variational continual learning[C]// Proceedings of the International Conference on Learning Representations.[S.l.:s.n.], 2020.
[50]	GORUR D . Linear mode connectivity in multitask and continual learning[J]. arXiv preprint,2020,arXiv:2010.04495.
[51]	KIRKPATRICK J , PASCANU R , RABINOWITZ N ,et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017,114(13): 3521-3526.
[52]	VERMA V K , LIANG K J , MEHTA N ,et al. Efficient feature transformations for discriminative and generative continual learning[J]. arXiv preprint,2021,arXiv:2103.13558.
[53]	CHA S , HSU H , HWANG T ,et al. CPR:classifier-projection regularization for continual learning[C]// Proceedings of the International Conference on Learning Representations.[S.l.:s.n.], 2020.
[54]	HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network[J]. arXiv preprint,2015,arXiv:1503.02531.
[55]	ZHAI M Y , CHEN L , TUNG F ,et al. Lifelong GAN:continual learning for conditional image generation[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 2759-2768.
[56]	TAO X Y , HONG X P , CHANG X Y ,et al. Few-shot class-incremental learning[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 12180-12189.
[57]	MENG Q E , SHIN'ICHI S ,, . ADINet:attribute driven incremental network for retinal image classification[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 4032-4041.
[58]	FINI E , LATHUILIèRE S , SANGINETO E ,et al. Online continual learning under extreme memory constraints[J]. Lecture Notes in Computer Science, 2020: 720-735.
[59]	SHMELKOV K , SCHMID C , ALAHARI K . Incremental learning of object detectors without catastrophic forgetting[C]// Proceedings of 2017 IEEE International Conference on Computer Vision.Piscataway:IEEE Press.[S.l.:s.n.], 2017: 3420-3429.
[60]	SMITH J , HSU Y C , BALLOCH J ,et al. Always be dreaming:a new approach for data-free class-incremental learning[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 9354-9364.
[61]	CASTRO F M , MARíN-JIMéNEZ M J , GUIL N ,et al. End-to-end incremental learning[J]. Lecture Notes in Computer Science, 2018: 241-257.
[62]	KIM J Y , CHOI D W . Split-and-bridge:adaptable class incremental learning within a single neural network[J]. arXiv preprint,2021,arXiv:2107.01349.
[63]	DENG D R , CHEN G Y , HAO J Y ,et al. Flattening sharpness for dynamic gradient projection memory benefits continual learning[J]. arXiv preprint,2021,arXiv:2110.04593.
[64]	TANG S X , CHEN D P , ZHU J G ,et al. Layerwise optimization by gradient decomposition for continual learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 9629-9638.
[65]	SAHA G , GARG I , ROY K . Gradient projection memory for continual learning[J]. arXiv preprint,2021,arXiv:2103.09762.
[66]	JIN X S , DU J Y , REN X . Gradient based memory editing for task-free continual learning[J]. arXiv preprint,2020,arXiv:2006.15294.
[67]	CAI Z P , SENER O , KOLTUN V . Online continual learning with natural distribution shifts:an empirical study with visual data[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 8261-8270.
[68]	CHAUDHRY A , KHAN N , DOKANIA P K ,et al. Continual learning in low-rank orthogonal subspaces[J]. Advances in Neural Information Processing Systems, 2020: 1-12.
[69]	LIN S , YANG L , FAN D L ,et al. TRGP:trust region gradient projection for continual learning[J]. arXiv preprint,2022,arXiv:2202.02931.
[70]	ALJUNDI R , KELCHTERMANS K , TUYTELAARS T . Task-free continual learning[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 11246-11255.
[71]	KIM C D , JEONG J , MOON S ,et al. Continual learning on noisy data streams via self-purified replay[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 517-527.
[72]	CHA H , LEE J , SHIN J . Co2L:contrastive continual learning[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 9496-9505.
[73]	KUZINA A , EGOROV E , BURNAEV E . BooVAE:boosting approach for continual learning of VAE[J]. Advances in Neural Information Processing Systems, 2021.
[74]	MARACANI A , MICHIELI U , TOLDO M ,et al. RECALL:replay-based continual learning in semantic segmentation[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 7006-7015.
[75]	AYUB A , WAGNER A R . EEC:learning to encode and regenerate images for continual learning[J]. arXiv preprint,2021,arXiv:2101.04904.
[76]	ROSTAMI M , KOLOURI S , PILLY P ,et al. Generative continual concept learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 5545-5552.
[77]	SHIN H , LEE J K , KIM J ,et al. Continual learning with deep generative replay[J]. arXiv preprint,2017,arXiv:1705.08690.
[78]	OSTAPENKO O , PUSCAS M , KLEIN T ,et al. Learning to remember:a synaptic plasticity driven framework for continual learning[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 11313-11321.
[79]	HU X T , TANG K H , MIAO C Y ,et al. Distilling causal effect of data in class-incremental learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 3956-3965.
[80]	LI T J , KE Q H , RAHMANI H ,et al. Else-Net:elastic semantic network for continual action recognition from skeleton data[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 13414-13423.
[81]	SINGH P , VERMA V K , MAZUMDER P ,et al. Calibrating CNNs for lifelong learning[J]. Advances in Neural Information Processing Systems, 2020.
[82]	PELLEGRINI L , GRAFFIETI G , LOMONACO V ,et al. Latent replay for real-time continual learning[C]// Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2020: 10203-10209.
[83]	KUNDU J N , VENKATESH R M , VENKAT N ,et al. Class-incremental domain adaptation[M]// Computer vision - ECCV 2020. Cham: Springer International Publishing, 2020: 53-69.
[84]	WEI K , DENG C , YANG X ,et al. Incremental embedding learning via zero-shot translation[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2021: 10254-10262.
[85]	STAN S , ROSTAMI M . Unsupervised model adaptation for continual semantic segmentation[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.], 2021: 2593-2601.
[86]	ZHU F , ZHANG X Y , WANG C ,et al. Prototype augmentation and self-supervision for incremental learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 5867-5876.
[87]	DE LANGE M , TUYTELAARS T . Continual prototype evolution:learning online from non-stationary data streams[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021: 8230-8239.
[88]	OLIVERI G , VAN LAAKE L C , CARISSIMO C ,et al. Continuous learning of emergent behavior in robotic matter[J]. Proceedings of the National Academy of Sciences of the United States of America, 2021,118(21): e2017015118.
[89]	YAN S P , XIE J W , HE X M . DER:dynamically expandable representation for class incremental learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 3013-3022.
[90]	VENIAT T , DENOYER L , RANZATO M . Efficient continual learning with modular networks and task-driven priors[C]// Proceedings of the 9th International Conference on Learning Representations.[S.l.:s.n.], 2021.
[91]	XU J , MA J , GAO X ,et al. Adaptive progressive continual learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[92]	CHEN T , ZHENYU Z , SIJIA L ,et al. Long live the lottery:the existence of winning tickets in lifelong learning[J]. International Conference on Learning Representations, 2021,51(4): 273-287.
[93]	SHI Y J , YUAN L , CHEN Y P ,et al. Continual learning via bit-level information preserving[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 16669-16678.
[94]	MAZUMDER P , SINGH P , RAI P . Few-shot lifelong learning[J]. arXiv preprint,2021,arXiv:2103.00991.
[95]	CACCIA M , RODRIGUEZ P , OSTAPENKO O ,et al. Online fast adaptation and knowledge accumulation:a new approach to continual learning[J]. Advances in Neural Information Processing Systems, 2020.
[96]	HURTADO J , RAYMOND-SAEZ A , SOTO A . Optimizing reusable knowledge for continual learning via metalearning[J]. Advances in Neural Information Processing Systems, 2021.
[97]	VON OSWALD J , ZHAO D , KOBAYASHI S ,et al. Learning where to learn:gradient sparsity in meta and continual learning[J]. arXiv preprint,2021,arXiv:2110.14402.
[98]	GUPTA G , YADAV K , PAULL L . La-MAML:look-ahead meta learning for continual learning[C]// Proceedings of the Advances in Neural Information Processing Systems.[S.l.:s.n.], 2020.
[99]	VOLPI R , LARLUS D , ROGEZ G . Continual adaptation of visual representations via domain randomization and meta-learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognitio. Piscataway:IEEE Press, 2021: 4441-4451.
[100]	LUO Y D , HUANG Z , ZHANG Z ,et al. Learning from the past:continual meta-learning with Bayesian graph neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 5021-5028.
[101]	ZHANG J , ZHANG J T , GHOSH S ,et al. Regularize,expand and compress:nonexpansive continual learning[C]// Proceedings of 2020 IEEE Winter Conference on Applications of Computer Vision. Piscataway:IEEE Press, 2020: 843-851.
[102]	EBRAHIMI S , ELHOSEINY M , DARRELL T ,et al. Uncertain ty-guided continual learning with Bayesian neural networks[J]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2019: 75-78.
[103]	RAVAGLIA L , RUSCI M , NADALINI D ,et al. A TinyML platform for on-device continual learning with quantized latent replays[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2021,11(4): 789-802.
[104]	PUTRA R V W , SHAFIQUE M . lpSpikeCon:enabling low-precision spiking neural network processing for efficient unsupervised continual learning on autonomous agents[J]. arXiv preprint,2022,arXiv:2205.12295.
[105]	BURHAN HAFEZ M , WERMTER S . Behavior self-organization supports task inference for continual robot learning[C]// Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE Press, 2021: 6739-6746.
[106]	XU J , ZHU Z . Reinforced continual learning[J]. Advances in Neural Information Processing Systems, 2018: 899-908.
[107]	LECARPENTIER E , ABEL D , ASADI K ,et al. Lipschitz lifelong reinforcement learning[C]// Proceedings of the AAAI Conference on Artificial Intelligenc.[S.l.:s.n.], 2021: 8270-8278.
[108]	BRAFMAN R I , TENNENHOLTZ M . R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning[J]. IJCAI International Joint Conference on Artificial Intelligence, 2001,3: 953-958.
[109]	ABEL D , JINNAI Y , YUE G ,et al. Policy and value transfer in lifelong reinforcement learning[C]// Proceedings of the 35th International Conference on Machine Learning.[S.l.:s.n.], 2018: 30-41.
[110]	LOMONACO V , DESAI K R , CULURCIELLO E ,et al. Continual reinforcement learning in 3D non-stationary environments[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE Press, 2020: 999-1008.
[111]	CHANDAK Y , THEOCHAROUS G , NOTA C ,et al. Lifelong learning with a changing action set[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(4): 3373-3380.
[112]	MNIH V , KAVUKCUOGLU K , SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.
[113]	ROLNICK D , AHUJA A , SCHWARZ J ,et al. Experience replay for continual learning[J]. arXiv preprint,2019,arXiv:1811.11682.
[114]	LU K , GROVER A , ABBEEL P ,et al. Reset-free lifelong Learning with skill-space planning[C]// Proceedings of the International Conference on Learning Representations.[S.l.:s.n.], 2020.
[115]	ALJUNDI R . Continual learning in neural networks[J]. arXiv preprint,2019,arXiv:1910.02718.
[116]	MORENO-MU?OZ P , RAMíREZ D , ARTéS-RODRíGUEZ A , . Continual learning for infinite hierarchical change-point detection[C]// Proceedings of 2020 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2020: 3582-3586.
[117]	BING L . Learning on the job:online lifelong and continual learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(9): 13544-13549.
[118]	FLESCH T , BALAGUER J , DEKKER R ,et al. Comparing continual task learning in minds and machines[J]. Proceedings of the National Academy of Sciences of the United States of America, 2018,115(44): 10313-10322.
[119]	BARRON H C . Neural inhibition for continual learning and memory[J]. Current Opinion in Neurobiology, 2021,67: 85-94.

机器人持续学习进展与展望

A review of continual learning for robotics

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 119

相关文章 15

Metrics

推荐阅读 0

[1]	苏震, 刘殿勇, 孙达智, 梁霄. 基于路径参数一致的水下机器人编队与避障控制[J]. 智能科学与技术学报, 2022, 4(4): 533-541.
[2]	蔡莹皓, 杨华, 安璇, 王文硕, 杜沂东, 张嘉韬, 王志刚. 神经符号学及其应用研究[J]. 智能科学与技术学报, 2022, 4(4): 560-570.
[3]	崔少伟, 王硕, 胡静怡, 张超凡. 面向机器人操作任务的视触觉传感技术综述[J]. 智能科学与技术学报, 2022, 4(2): 186-199.
[4]	徐德, 秦方博. 机器人自动轴孔装配研究进展[J]. 智能科学与技术学报, 2022, 4(2): 200-211.
[5]	葛悦光, 张少林, 蔡莹皓, 鲁涛, 温大勇, 王海涛, 王硕. 本体知识表示方法在机器人领域的应用研究综述[J]. 智能科学与技术学报, 2022, 4(2): 212-222.
[6]	余跃, 刘欣, 蒋芳清, 张晗, 王晖, 曾炜. 支持鹏程系列开源大模型应用生态演化的可持续学习能力探索[J]. 智能科学与技术学报, 2022, 4(1): 97-108.
[7]	赵航, 刘胜, 罗坤, 陈世超, 孔令辉, 贾凡. 面向KubeEdge边缘计算系统应用研究[J]. 智能科学与技术学报, 2022, 4(1): 118-128.
[8]	王腾, 潘晶, 董璐, 孙长银. 面向防疫的智能导诊机器人关键技术及应用[J]. 智能科学与技术学报, 2021, 3(2): 187-194.
[9]	邓晓燕, 俞祝良, 林灿光, 顾正晖, 李远清. 基于脑机接口的人机共享控制技术研究[J]. 智能科学与技术学报, 2021, 3(1): 85-92.
[10]	傅汇乔, 唐开强, 邓归洲, 王鑫鹏, 陈春林. 基于深度强化学习的六足机器人运动规划[J]. 智能科学与技术学报, 2020, 2(4): 361-371.
[11]	王大伟,王卓,王鹏,王志刚,王海涛. 基于边缘计算的云原生机器人系统[J]. 智能科学与技术学报, 2020, 2(3): 275-283.
[12]	付海军,陈世超,林懿伦,熊刚,胡斌. 人在回路的混合增强智能在Sawyer的研究与验证[J]. 智能科学与技术学报, 2019, 1(3): 280-286.
[13]	徐天成,王雪军,卢东东,卢梦叶,林祺,张小强,成艺. 智能针灸机器人关键技术及发展趋势[J]. 智能科学与技术学报, 2019, 1(3): 305-310.
[14]	白天翔,沈震,刘雅婷,董西松. 平行机器：一种智能机器的管理与控制框架[J]. 智能科学与技术学报, 2019, 1(2): 181-191.
[15]	毛翊超,陆麒. 基于视觉伺服技术的工业机器人高动态物体跟踪系统[J]. 智能科学与技术学报, 2019, 1(1): 62-69.