行人轨迹预测方法综述

doi:10.11959/j.issn.2096-6652.202140

智能科学与技术学报 ›› 2021, Vol. 3 ›› Issue (4): 399-411.doi: 10.11959/j.issn.2096-6652.202140

行人轨迹预测方法综述

李琳辉¹^,², 周彬¹, 任威威¹, 连静¹^,²

¹ 大连理工大学汽车工程学院，辽宁大连 116024
² 大连理工大学工业装备结构分析国家重点实验室，辽宁大连 116024

修回日期:2020-08-28 出版日期:2021-12-15 发布日期:2021-12-01
作者简介:李琳辉（1981- ），男，博士，大连理工大学汽车工程学院副教授，主要研究方向为智能车辆环境感知、规划决策与导航控制等
周彬（1997- ），男，大连理工大学汽车工程学院硕士生，主要研究方向为智能车辆规划决策、轨迹预测等
任威威（1996- ），男，大连理工大学汽车工程学院硕士生，主要研究方向为智能车辆规划决策、轨迹预测等
连静（1980- ），女，博士，大连理工大学汽车工程学院副教授，主要研究方向为新能源汽车智能化、轨迹预测等
基金资助:
国家自然科学基金资助项目(51775082);国家自然科学基金资助项目(61976039);中央高校基本科研业务费专项资金资助项目(DUT19LAB36);中央高校基本科研业务费专项资金资助项目(DUT20GJ207);大连市科技创新基金资助项目(2018J12GX061)

Review of pedestrian trajectory prediction methods

Linhui LI¹^,², Bin ZHOU¹, Weiwei REN¹, Jing LIAN¹^,²

¹ School of Automotive Engineering, Dalian University of Technology, Dalian 116024, China
² State Key Laboratory of Structural Analysis for Industrial Equipment, Dalian University of Technology, Dalian 116024, China

Revised:2020-08-28 Online:2021-12-15 Published:2021-12-01
Supported by:
The National Natural Science Foundation of China(51775082);The National Natural Science Foundation of China(61976039);The China Fundamental Research Funds for the Central Universities(DUT19LAB36);The China Fundamental Research Funds for the Central Universities(DUT20GJ207);Science and Technology Innovation Fund of Dalian(2018J12GX061)

摘要/Abstract

摘要：

随着深度学习技术的突破和大型数据集的提出，行人轨迹预测的准确度已经成为人工智能领域的研究热点之一。主要对行人轨迹预测的技术分类和研究现状进行详细的综述。根据模型建模方式的不同，将现有方法分为基于浅层学习的轨迹预测方法和基于深度学习的轨迹预测方法，分析了每类方法中具有代表性的算法的效果及优缺点，归纳了当前主流的轨迹预测公开数据集，并在数据集中对比了主流轨迹预测方法的性能，最后对轨迹预测技术面临的挑战与发展趋势进行了展望。

关键词: 轨迹预测, 深度学习, 序列决策

Abstract:

With the breakthrough of deep learning technology and the proposal of large data sets, the accuracy of pedestrian trajectory prediction has become one of the research hotspots in the field of artificial intelligence.The technical classification and research status of pedestrian trajectory prediction were mainly reviewed.According to the different modeling methods, the existing methods were divided into shallow learning and deep learning based trajectory prediction algorithms, the advantages and disadvantages of representative algorithms in each type of method were analyzed and introduced.Then, the current mainstream public data sets were summarized, and the performance of mainstream trajectory prediction methods based on the data sets was compared.Finally, the challenges faced by the trajectory prediction technology and the development direction of future work were prospected.

Key words: trajectory prediction, deep learning, sequence decision

中图分类号:

TP520

李琳辉, 周彬, 任威威, 等. 行人轨迹预测方法综述[J]. 智能科学与技术学报, 2021, 3(4): 399-411.

Linhui LI, Bin ZHOU, Weiwei REN, et al. Review of pedestrian trajectory prediction methods[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 399-411.

图/表 6

图1

图2

图3

图4

图5

表1

参考文献 51

[1]	RUDENKO A , PALMIERI L , HERMAN M ,et al. Human motion trajectory prediction:a survey[J]. The International Journal of Robotics Research, 2020(1):027836492091744.
[2]	DAI M M , WANG J X , YIN G D ,et al. Dynamic output-feedback robust control for vehicle path tracking considering different human drivers’ characteristics[C]// Proceedings of the 36th Chinese Control Conference. Piscataway:IEEE Press, 2017: 9407-9412.
[3]	MOUSSAID M , PEROZO N , GARNIER S ,et al. The walking behaviour of pedestrian social groups and its impact on crowd dynamics[J]. PLoS ONE, 2010,5(4): e10047.
[4]	LEFEVRE S , VASQUEZ D , LAUGIER C . A survey on motionprediction and risk assessment for intelligent vehicles[J]. Robomech Journal, 2014,1(1): 1-14.
[5]	KELLER C G , GAVRILA D M . Will the pedestrian cross? A study onpedestrian path prediction[J]. IEEE Transactions on Intelligent Transportation Systems, 2014,15(2): 494-506.
[6]	SCHNEIDER N , GAVRILA D M . Pedestrian path prediction with recursive Bayesian filters:a comparative study[C]// Proceedings of the 35th German Conference on Pattern Recognition. Berlin:Springer Press, 2013: 174-183.
[7]	PAVLOVIC V , REHG J M , MACCORMICK J . Learning switching linear models of human motion[C]// Proceeding of the Conference and Workshop on Neural Information Processing Systems. New York:Curran Associates Press, 2000: 981-987.
[8]	FOX E B , SUDDERTH E B , JORDAN M ,et al. Bayesian nonparametric inference of switching dynamic linear models[J]. IEEE Transactions on Signal Processing, 2011,59(4): 1569-1585.
[9]	KOOIJ J F P , SCHNEIDER N , FLOHR F ,et al. Context-based pedestrian path prediction[C]// Proceedings of the 13th European Conference on Computer Vision. Berlin:Springer Press, 2014: 618-633.
[10]	HELBING D , MOLNAR P . Social force model for pedestrian dynamics[J]. Physical Review E, 1995,51(5): 4282-4286.
[11]	BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate[J]. Computer Science, 2014.
[12]	ALAHI A , RAMANATHAN V , LI F F . Socially-aware large-scale crowd forecasting[C]// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 2211-2218.
[13]	YI S , LI H S , WANG X G . Understanding pedestrian behaviors from stationary crowd groups[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 3488-3496.
[14]	GOLI S A , FAR B H , FAPOJUWO A O . Vehicle trajectory prediction with Gaussian process regression in connected vehicle environment[C]// Proceedings of 2018 IEEE Intelligent Vehicles Symposium. Piscataway:IEEE Press, 2018: 550-555.
[15]	ELLIS D , SOMMERLADE E , REID I . Modelling pedestrian trajectory patterns with Gaussian processes[C]// Proceedings of 2009 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2009: 1229-1234.
[16]	RASMUSSEN C E , WILLIAMS C K I . Gaussian processes for machine learning[M]. Cambridge: MIT Press, 2005.
[17]	HOCHREITER S , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[18]	CHUNG J Y , GULCEHRE C , CHO K ,et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. Eprint Arxiv, 2014.
[19]	GRAVES A , JAITLY N . Towards end-to-end speech recognition with recurrent neural networks[C]// Proceedings of the 31st International Conference on Machine Learning. New York:ACM Press, 2014: 1764-1772.
[20]	CHOROWSKI J , BAHDANAU D , CHO K ,et al. End-to-end continuous speech recognition using attention-based Recurrent NN:first results[J]. arXiv preprint,2014,arXiv:1412. 1602.
[21]	DONAHUE J , HENDRICKS L A , GUADARRAMA S ,et al. Long-term recurrent convolutional networks for visual recognition and description[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 2625-2634.
[22]	WU H , CHEN Z Y , SUN W W ,et al. Modeling trajectories with recurrent neural networks[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. Menlo Park:AAAI Press, 2017: 3083-3090.
[23]	KARATZOLOU A , JABLONSKI A , BEIGL M . A Seq2Seq learning approach for modeling semantic trajectories and predicting the next location[C]// Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York:ACM Press, 2018: 528-531.
[24]	ALAHI A , GOEL K , RAMANATHAN V ,et al. Social LSTM:human trajectory prediction in crowded spaces[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 961-971.
[25]	KITANI K M , ZIEBART B D , BAGNELL J A ,et al. Activity forecasting[C]// Proceedings of the 12th European Conference on Computer Vision. Berlin:Springer Press, 2012: 201-214.
[26]	XUE H , HUYNH D Q , REYNOLDS M . SS-LSTM:a hierarchical LSTM model for pedestrian trajectory prediction[C]// Proceedings of 2018 IEEE Workshop on Applications of Computer Vision. Piscataway:IEEE Press, 2018: 1186-1194.
[27]	GOODFELLOW I J , POUGET-ABADIE J ,, MIRZA M , et al . Generative adversarial nets[C]// Proceedings of the Conference and Workshop on Neural Information Processing Systems. New York:Curran Associates Press, 2014: 2672-2680.
[28]	KIPF T N , WELLING M . Variational graph auto-encoders[J]. arXiv preprint,2016,arXiv:1611.07308.
[29]	GUPTA A , JOHNSON J , LI F F ,et al. Social GAN:socially acceptable trajectories with generative adversarial networks[C]// Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 2255-2264.
[30]	AMIRIAN J , HAYET J B , PETTRE J . Social ways:learning multi-modal distributions of pedestrian trajectories with GANs[J]. arXiv preprint,2019,arXiv:1904.09507.
[31]	VARSHNEYA D , SRINIVASARAGHAVAN G . Human trajectory prediction using spatially aware deep attention models[J]. arXiv preprint,2017,arXiv:1705.09436.
[32]	SADEGHIAN A , KOSARAJU V , SADEGHIAN A ,et al. Sophie:an attentive GAN for predicting paths compliant to social and physical constraints[C]// Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 1349-1358.
[33]	KOSARAJU V , SADEGHIAN A , MARTIN-MARTIN R ,et al. Social-BiGAT:multimodal trajectory forecasting using bicycle-GAN and graph attention networks[C]// Proceedings of the Conference and Workshop on Neural Information Processing Systems. New York:Curran Associates Press, 2019: 137-146.
[34]	CHENG H , YANG W L M Y , SESTER M ,et al. Context conditional variational autoencoder for predicting multi-path trajectories in mixed traffic[J]. arXiv preprint,2020,arXiv:2002.05966.
[35]	YANG B , YAN G , WANG P ,et al. TPPO:a novel trajectory predictor with pseudo oracle[J]. arXiv preprint, 2020.
[36]	LIANG J W , JIANG L , NIEBLES J C ,et al. Peeking into the future:predicting future person activities and locations in videos[C]// Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 5725-5734.
[37]	SUN J H , JIANG Q H , LU C W . Recursive social behavior graph for trajectory prediction[J]. arXiv preprint,2020,arXiv:2004.10402.
[38]	LI G H , MULLER M , QIAN G C ,et al. DeepGCNs:can GCNs go as deep as CNNs?[C]// Proceedings of 2019 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 9267-9276.
[39]	KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks[J]. arXiv preprint,2016,arXiv:1609.02907.
[40]	SCHOLKOPF B , TSUDA K , VERT J ,et al. Kernel methods in computational biology[M]. Cambridge: MIT Press, 2004.
[41]	SCHLICHTKRULL M , KIPF T N , BLOEM P ,et al. Modeling relational data with graph convolutional networks[C]// Proceedings of the European Semantic Web Conference. Berlin:Springer Press, 2018: 593-607.
[42]	BERG R V D , KIPF T N , WELLING M . Graph convolutional matrix completion[J]. arXiv preprint,2017,arXiv:1706.02263.
[43]	ZHANG L D , SHE Q , GUO P . Stochastic trajectory prediction with social graph network[J]. arXiv preprint,2019,arXiv:1907.10233.
[44]	YAN S J , XIONG Y J , LIN D H ,et al. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]// Proceedings of the National Conference on Artificial Intelligence. Menlo Park:AAAI Press, 2018: 7444-7452.
[45]	HADDAD S , WU M Q , WEI H ,et al. Situation-aware pedestrian trajectory prediction with spatio-temporal attention model[J]. arXiv preprint,2019,arXiv:1902.05437.
[46]	MOHAMED A , QIAN K , ELHOSEINY M ,et al. Social-STGCNN:a social spatio-temporal graph convolutional neural network for human trajectory prediction[J]. arXiv preprint,2020,arXiv:2002.11927.
[47]	PELLEGRINI S , ESS A , GOOL L V . Improving data association by joint modeling of pedestrian trajectories and groupings[C]// Proceedings of the 11th European Conference on Computer Vision. Berlin:Springer Press, 2010: 452-465.
[48]	LERNER A , CHRYSANTHOU Y , LISCHINSKI D . Crowds by example[J]. Computer Graphics Forum, 2007,26(3): 655-664.
[49]	ROBICQUET A , SADEGHIAN A , ALAHI A ,et al. Learning social etiquette:human trajectory prediction in crowded scenes[C]// Proceedings of the 14th European Conference on Computer Vision. Berlin:Springer Press, 2016: 549-565.
[50]	VEMULA A , MUELLING K , OH J . Social attention:modeling attention in human crowds[C]// Proceedings of 2018 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2018: 4601-4607.
[51]	HUANG Y F , BI H K , LI Z X ,et al. STGAT:Modeling spatial-temporal interactions for human trajectory prediction[C]// Proceedings of 2019 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 6272-6281.

方法	参考文献序号	评价指标（ADE值/FDE值）
方法	参考文献序号	ETH	Hotel	Univ	Zara1	Zara2	平均值
Linear	[24]	1.33/2.94	0.39/0.72	0.82/1.59	0.62/1.21	0.77/1.48	0.79/1.59
S-LSTM	[24]	1.09/2.35	0.79/1.76	0.67/1.40	0.47/1.00	0.56/1.17	0.72/1.54
O-LSTM	[24]	1.05/2.21	0.81/1.68	0.71/1.45	0.47/1.02	0.64/1.25	0.74/1.52
S-RNN	[45]	2.72/4.60	0.85/1.35	1.05/2.20	1.60/3.50	1.45/3.00	1.53/2.93
Social-attention	[50]	3.60/4.70	0.79/1.44	1.30/2.66	0.95/2.05	1.00/2.14	1.53/3.52
S-GAN	[29]	0.81/1.52	0.72/1.61	0.60/1.26	0.34/0.69	0.42/0.84	0.58/1.18
S-GAN-P	[29]	0.87/1.62	0.67/1.37	0.76/1.52	0.35/0.68	0.42/0.84	0.61/1.21
Sophie	[32]	0.70/1.43	0.76/1.67	0.54/1.24	0.30/0.63	0.38/0.78	0.54/1.15
Social-ways	[30]	0.39/0.64	0.39/0.66	0.55/1.31	0.44/0.64	0.51/0.92	0.46/0.82
GAT	[33]	0.68/1.29	0.68/1.40	0.57/1.29	0.29/0.60	0.37/0.75	0.52/1.07
STGAT	[51]	0.70/1.35	0.37/0.67	0.59/1.23	0.35/0.69	0.31/0.64	0.47/0.92
Social-BiGAT	[33]	0.69/1.29	0.49/1.01	0.55/1.32	0.30/0.62	0.36/0.75	0.48/1.00
NEXT	[36]	0.73/1.65	0.30/0.59	0.60/1.27	0.38/0.81	0.31/0.68	0.46/1.00
TPNet-20	[37]	0.84/1.73	0.24/0.46	0.42/0.94	0.33/0.75	0.26/0.60	0.42/0.90
Social-STGCNN	[46]	0.64/1.11	0.79/0.85	0.44/0.79	0.34/0.53	0.30/0.48	0.44/0.75

行人轨迹预测方法综述

Review of pedestrian trajectory prediction methods

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 51

相关文章 15

Metrics

推荐阅读 0

[1]	黄哲, 王永才, 李德英. 3D目标检测方法研究综述[J]. 智能科学与技术学报, 2023, 5(1): 7-31.
[2]	卢经纬, 程相, 王飞跃. 求解微分方程的人工智能与深度学习方法：现状及展望[J]. 智能科学与技术学报, 2022, 4(4): 461-476.
[3]	张俊, 许沛东, 陈思远, 高天露, 戴宇欣, 张科, 赵杭, 高杰迈, 白昱阳, 李金星, 张浩然, 李湘, 陈玖香. 物理-数据-知识混合驱动的人机混合增强智能系统管控方法[J]. 智能科学与技术学报, 2022, 4(4): 571-583.
[4]	陈妍, 罗雪琴, 梁伟, 谢永芳. 基于情感信息融合注意力机制的抑郁症识别[J]. 智能科学与技术学报, 2022, 4(4): 600-609.
[5]	栗仁武, 张凌霄, 高林, 李淳芃, 蒋浩. 基于点云的类级别物体姿态估计[J]. 智能科学与技术学报, 2022, 4(2): 246-254.
[6]	李颖, 陈龙, 黄钊宏, 孙杨, 蔡国榕. 基于多尺度卷积神经网络特征融合的植株叶片检测技术[J]. 智能科学与技术学报, 2021, 3(3): 304-311.
[7]	田庆, 胡蓉, 李佐勇, 蔡远征, 余兆钗. 基于SE-YOLOv5s的绝缘子检测[J]. 智能科学与技术学报, 2021, 3(3): 312-321.
[8]	刘文, 胡琨林, 李岩, 刘钊. 移动目标轨迹预测方法研究综述[J]. 智能科学与技术学报, 2021, 3(2): 149-160.
[9]	张阳, 胡月, 辛东嵘. 一种考虑时空关联的深度学习短时交通流预测方法[J]. 智能科学与技术学报, 2021, 3(2): 172-178.
[10]	刘栋军, 王宇涵, 凌文芬, 彭勇, 孔万增. 基于脑机协同智能的情绪识别[J]. 智能科学与技术学报, 2021, 3(1): 65-75.
[11]	赵亮, 谢志峰, 张坤鹏, 郑玉卿, 付园坤. 无线网络信号传输建模：一种区间二型模糊集成深度学习方法[J]. 智能科学与技术学报, 2020, 2(4): 401-411.
[12]	田思佳,顾强,胡蓉,李锐戈,何顶新. 一种基于深度学习的机械臂分拣方法[J]. 智能科学与技术学报, 2020, 2(3): 268-274.
[13]	王飞跃,曹东璞,魏庆来. 强化学习：迈向知行合一的智能机制与算法[J]. 智能科学与技术学报, 2020, 2(2): 101-106.
[14]	袁小锋,王雅琳,阳春华,桂卫华. 深度学习在流程工业过程数据建模中的应用[J]. 智能科学与技术学报, 2020, 2(2): 107-115.
[15]	刘凯, 李浥东, 林伟鹏. 车辆再识别技术综述[J]. 智能科学与技术学报, 2020, 2(1): 10-25.