基于社会注意力机制的行人轨迹预测方法研究

doi:10.11959/j.issn.1000-436x.2020100

通信学报 ›› 2020, Vol. 41 ›› Issue (6): 175-183.doi: 10.11959/j.issn.1000-436x.2020100

基于社会注意力机制的行人轨迹预测方法研究

李琳辉^1,²,周彬¹,连静^1,²(),周雅夫¹

¹ 大连理工大学汽车工程学院，辽宁大连 116024
² 大连理工大学工业装备结构分析国家重点实验室，辽宁大连 116024

修回日期:2020-03-13 出版日期:2020-06-25 发布日期:2020-07-04
作者简介:李琳辉（1981- ），男，河南辉县人，博士，大连理工大学副教授，主要研究方向为智能车辆环境感知、规划决策与导航控制等|周彬（1997- ），男，山东临沂人，大连理工大学硕士生，主要研究方向为智能车辆规划决策、轨迹预测等|连静（1980- ），女，吉林公主岭人，博士，大连理工大学副教授，主要研究方向为新能源汽车智能化、轨迹预测等|周雅夫（1962- ），男，辽宁大连人，大连理工大学教授，主要研究方向为新能源汽车动力控制、新能源汽车网联化等
基金资助:
国家自然科学基金资助项目(61976039);国家自然科学基金资助项目(51775082);中央高校基本科研业务费专项基金资助项目(DUT19LAB36);中央高校基本科研业务费专项基金资助项目(DUT17LAB11)

Research on pedestrian trajectory prediction method based on social attention mechanism

Linhui LI^1,²,Bin ZHOU¹,Jing LIAN^1,²(),Yafu ZHOU¹

¹ School of Automotive Engineering,Dalian University of Technology,Dalian 116024,China
² State Key Laboratory of Structural Analysis for Industrial Equipment,Dalian University of Technology,Dalian 116024,China

Revised:2020-03-13 Online:2020-06-25 Published:2020-07-04
Supported by:
The National Natural Science Foundation of China(61976039);The National Natural Science Foundation of China(51775082);The Fundamental Research Funds for the Central Universities(DUT19LAB36);The Fundamental Research Funds for the Central Universities(DUT17LAB11)

摘要/Abstract

摘要：

为提高行人交互中轨迹预测速度、精度与模型可解释性，提出了一种基于社会注意力机制的GAN模型。首先，定义了一种新型社会关系，对行人间的影响进行社会关系建模，设计了基于注意力机制的网络模型，提高了网络预测速度和可解释性。然后，探索不同池化汇集机制对预测结果的影响，确定性能优异的池化模型。最后，搭建了轨迹预测网络，并在UCY和ETH数据集中进行训练。实验结果表明，所提模型预测精度优于现有方法，且实时性较现有方法提升18.3% 。

关键词: 行人轨迹预测, 生成对抗网络, 注意力机制, 社会力模型, 最优池化模型

Abstract:

In order to improve the speed,accuracy and model interpretability of trajectory prediction in pedestrian interaction,a GAN model based on social attention mechanism was proposed.Firstly,a new type of social relationship on pedestrians was defined to model social relationships and a network model based on the attention mechanism was designed to improve the speed and interpretability of network prediction.Secondly,the influence of different pooling mechanisms on the prediction results was explored to determine the pooling model with excellent performance.Finally,a trajectory prediction network was built on this basis and trained on the UCY and ETH data sets.The experimental results show that the model not only has better prediction accuracy than the existing methods,but also improves the real-time performance by 18.3% compared with the existing methods.

Key words: pedestrian trajectory prediction, generative adversarial network, attention mechanism, social force model, optimal pooling model

中图分类号:

TP391

李琳辉,周彬,连静,周雅夫. 基于社会注意力机制的行人轨迹预测方法研究[J]. 通信学报, 2020, 41(6): 175-183.

Linhui LI,Bin ZHOU,Jing LIAN,Yafu ZHOU. Research on pedestrian trajectory prediction method based on social attention mechanism[J]. Journal on Communications, 2020, 41(6): 175-183.

图/表 5

图1

表1

表2

图2

表3

参考文献 24

[1]	DAI M , WANG J , YIN G ,et al. Dynamic output-feedback robust control for vehicle path tracking considering different human drivers' characteristics[C]// Technical Committee on Control Theory. Piscataway:IEEE Press, 2017: 1157-1162.
[2]	MEHDI M , NIRIASKA P , SIMON G ,et al. The walking behaviour of pedestrian social groups and its impact on crowd dynamics[J]. Plos One, 2010,5(4):e10047.
[3]	KITANI K , ZIEBART B , BAGNELL J A ,et al. Activity forecasting[C]// Proceedings of the 12th European conference on Computer Vision - Volume Part IV. Berlin:Springer, 2012: 201-214.
[4]	LEE N , CHOI W , VERNAZA P ,et al. Desire:distant future prediction in dynamic scenes with interacting agents[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 336-345.
[5]	YAMAGUCHI K , BERG A C , ORTIZ L E ,et al. Who are you with and where are you going?[C]// Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2011: 1345-1352.
[6]	HOCHREITER S , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[7]	VEMULA A , MUELLING K , OH J . Social attention:modeling attention in human crowds[C]// 2018 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2018: 1-7.
[8]	ALAHI A , GOEL K , RAMANATHAN V ,et al. Social LSTM:human trajectory prediction in crowded spaces[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 961-971.
[9]	GUPTA A , JOHNSON J,FEI-FEI L , et al . Social GAN:socially acceptable trajectories with generative adversarial networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 2255-2264.
[10]	GOODFELLOW I , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial nets[C]// Advances in Neural Information Processing Systems. Cambridge:MIT Press, 2014: 2672-2680.
[11]	SADEGHIAN A , KOSARAJU V , SADEGHIAN A ,et al. SoPhie:an attentive GAN for predicting paths compliant to social and physical constraints[J]. ArXiv Preprint,ArXiv:1806.01482, 2018
[12]	VINEET K , AMIR S , ROBERTO M ,et al. Social-BiGAT:multimodal trajectory forecasting using bicycle-GAN and graph attention networks[J]. ArXiv Preprint ArXiv:1907.03395, 2019
[13]	PETAR V , GUILLEM C , ARANTXA C ,et al. Graph attention networks[J]. CoRR,abs/1710.10903, 2018
[14]	MNIH V , HEESS N , GRAVES A ,et al. Recurrent models of visual attention[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014: 2204-2212.
[15]	KINGMA D P , WELLING M . Auto-encoding variational Bayes[J]. ArXiv preprint,ArXiv:1312.6114, 2013
[16]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Advances in Neural Information Processing Systems. Cambridge:MIT Press, 2017: 5998-6008.
[17]	孙亚圣, 姜奇, 胡洁 ,等. 基于注意力机制的行人轨迹预测生成模型[J]. 计算机应用, 2019,39(3): 668-674.
	SUN Y S , JIANG Q , HU J ,et al. Attention mechanism based pedestrian trajectory prediction generation model[J]. Journal of Computer Applications, 2019,39(3): 668-674.
[18]	XU K , BA J , KIROS R ,et al. Show,attend and tell:neural image caption generation with visual attention[J]. ArXiv Preprint,ArXiv:1502.03044, 2015
[19]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. ArXiv Preprint,ArXiv:1411.1784, 2014
[20]	XU Q , HUANG G , YUAN Y ,et al. An empirical study on evaluation metrics of generative adversarial networks[J]. ArXiv Preprint,ArXiv:1806.07755, 2018
[21]	PELLEGRINI S , ESS A , VAN GOOL L . Improving data association by joint modeling of pedestrian trajectories and groupings[C]// European Conference on Computer Vision. Berlin:Springer, 2010: 452-465.
[22]	LERNER A , CHRYSANTHOU Y , LISCHINSKI D . Crowds by example[J]. Computer Graphics Forum, 2007,26(3): 655-664.
[23]	ABDUALLAH M , KUN Q , MOHAMED E ,et al. Social-STGCNN:a social spatio-temporal graph convolutional neural network for human trajectory prediction[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020.
[24]	HE Z , CHOW C , ZHANG J . STCNN:a spatio-temporal convolutional neural network for long-term traffic prediction[C]// 2019 20th IEEE International Conference on Mobile Data Management. Piscataway:IEEE Press, 2019: 226-233.

数据集			ADE/m				FDE/m
数据集	Gumbel Pooling	Max Pooling	Average Pooling	Random Pooling	Gumbel Pooling	Max Pooling	Average Pooling	Random Pooling
ETH	0.58	0.55	0.64	0.60	1.17	1.05	1.24	1.14
Hotel	0.31	0.45	0.29	0.34	0.62	0.86	0.55	0.65
Univ	0.34	0.52	0.37	0.37	0.69	0.93	0.75	0.74
Zara1	0.21	0.54	0.22	0.21	0.45	0.98	0.42	0.41
Zara2	0.21	0.42	0.22	0.21	0.43	0.72	0.44	0.42
平均	0.33	0.496	0.348	0.346	0.672	0.908	0.68	0.672

预测时间	数据集				评价基准ADE/FDE
预测时间	数据集	Linear	LSTM	S-LSTM	S-GAN	Sophie	Social-BiGAT	SA-GAN
	ETH	0.84/1.60	0.70/1.45	0.73/1.48	0.67/1.47	—	—	0.58/1.17
	Hotel	0.35/0.60	0.55/1.17	0.49/1.01	0.52/1.05	—	—	0.31/0.62
t_pred=8	Univ	0.56/1.01	0.36/0.77	0.41/0.84	0.34/0.75	—	—	0.34/0.69
	Zara1	0.41/0.74	0.25/0.74	0.27/0.56	0.26/0.45	—	—	0.21/0.45
	Zara2	0.53/0.95	0.31/0.65	0.33/0.79	0.29/0.58	—	—	0.21/0.43
	平均	0.53/0.98	0.43/0.91	0.45/0.91	0.42/0.86	—	—	0.33/0.67
	ETH	1.33/2.94	1.09/2.41	1.09/2.35	0.92/1.73	0.70/1.43	0.69/1.29	0.72/1.28
	Hotel	0.39/0.72	0.86/1.91	0.79/1.76	0.67/1.37	0.76/1.67	0.49/1.01	0.50/1.01
t_pred=12	Univ	0.82/1.59	0.61/1.31	0.67/1.40	0.76/1.52	0.54/1.24	0.55/1.32	0.58/1.19
	Zara1	0.62/1.21	0.41/0.88	0.47/1.00	0.35/0.68	0.30/0.63	0.30/0.62	0.42/0.83
	Zara2	0.77/1.48	0.52/1.11	0.56/1.17	0.42/0.84	0.38/0.78	0.36/0.75	0.39/0.85
	平均	0.79/1.59	0.70/1.52	0.72/1.54	0.62/1.23	0.54/1.15	0.48/1.00	0.52/1.03

预测模型	预测8帧时间/min	预测12帧时间/min
LSTM	0.029	0.042
S-LSTM	0.504	0.825
S-GAN	0.202	0.341
SA-GAN	0.163	0.282

基于社会注意力机制的行人轨迹预测方法研究

Research on pedestrian trajectory prediction method based on social attention mechanism

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 24

相关文章 15

Metrics

推荐阅读 0

[1]	张佳乐, 朱诚诚, 孙小兵, 陈兵. 基于GAN的联邦学习成员推理攻击与防御方法[J]. 通信学报, 2023, 44(5): 193-205.
[2]	苏新, 张桂福, 行鸿彦, Zenghui Wang. 基于平衡生成对抗网络的海洋气象传感网入侵检测研究[J]. 通信学报, 2023, 44(4): 124-136.
[3]	刘延华, 李嘉琪, 欧振贵, 高晓玲, 刘西蒙, MENG Weizhi, 刘宝旭. 对抗训练驱动的恶意代码检测增强方法[J]. 通信学报, 2022, 43(9): 169-180.
[4]	王延文, 雷为民, 张伟, 孟欢, 陈新怡, 叶文慧, 景庆阳. 基于生成模型的视频图像重建方法综述[J]. 通信学报, 2022, 43(9): 194-208.
[5]	李昂, 陈建新, 魏昕, 周亮. 面向6G的跨模态信号重建技术[J]. 通信学报, 2022, 43(6): 28-40.
[6]	段雪源, 付钰, 王坤. 基于VAE-WGAN的多维时间序列异常检测方法[J]. 通信学报, 2022, 43(3): 1-13.
[7]	冯海林, 张潇, 刘同存. 融合评论文本特征和评分图卷积表示的推荐模型[J]. 通信学报, 2022, 43(3): 164-171.
[8]	向夏雨, 王佳慧, 王子睿, 段少明, 潘鹤中, 庄荣飞, 韩培义, 刘川意. 基于生成对抗网络技术的医疗仿真数据生成方法[J]. 通信学报, 2022, 43(3): 211-224.
[9]	仲美玉, 吴培良, 窦燕, 刘毅, 孔令富. 基于中文语义-音韵信息的语音识别文本校对模型[J]. 通信学报, 2022, 43(11): 65-79.
[10]	陈卓, 朱淼, 杜军威. 基于多视角图神经网络的欺诈检测算法[J]. 通信学报, 2022, 43(11): 225-232.
[11]	陆彦辉, 柳寒, 李航, 朱光旭. 基于多鉴别器生成对抗网络的时间序列生成模型[J]. 通信学报, 2022, 43(10): 167-176.
[12]	刘威, 陈成, 江锐, 卢涛. 四通道无监督学习图像去雾网络[J]. 通信学报, 2022, 43(10): 210-222.
[13]	王洪雁, 袁海. 基于骨骼及表观特征融合的动作识别方法[J]. 通信学报, 2022, 43(1): 138-148.
[14]	周志立, 王美民, 杨高波, 朱剑宇, 孙星明. 基于轮廓自动生成的构造式图像隐写方法[J]. 通信学报, 2021, 42(9): 144-154.
[15]	邹福泰, 谭越, 王林, 蒋永康. 基于生成对抗网络的僵尸网络检测[J]. 通信学报, 2021, 42(7): 95-106.