基于社会注意力机制的行人轨迹预测方法研究

doi:10.11959/j.issn.1000-436x.2020100

Abstract

Abstract:

In order to improve the speed,accuracy and model interpretability of trajectory prediction in pedestrian interaction,a GAN model based on social attention mechanism was proposed.Firstly,a new type of social relationship on pedestrians was defined to model social relationships and a network model based on the attention mechanism was designed to improve the speed and interpretability of network prediction.Secondly,the influence of different pooling mechanisms on the prediction results was explored to determine the pooling model with excellent performance.Finally,a trajectory prediction network was built on this basis and trained on the UCY and ETH data sets.The experimental results show that the model not only has better prediction accuracy than the existing methods,but also improves the real-time performance by 18.3% compared with the existing methods.

Key words: pedestrian trajectory prediction, generative adversarial network, attention mechanism, social force model, optimal pooling model

CLC Number:

TP391

Linhui LI,Bin ZHOU,Jing LIAN,Yafu ZHOU. Research on pedestrian trajectory prediction method based on social attention mechanism[J]. Journal on Communications, 2020, 41(6): 175-183.

Figures/Tables 5

References 24

[1]	DAI M , WANG J , YIN G ,et al. Dynamic output-feedback robust control for vehicle path tracking considering different human drivers' characteristics[C]// Technical Committee on Control Theory. Piscataway:IEEE Press, 2017: 1157-1162.
[2]	MEHDI M , NIRIASKA P , SIMON G ,et al. The walking behaviour of pedestrian social groups and its impact on crowd dynamics[J]. Plos One, 2010,5(4):e10047.
[3]	KITANI K , ZIEBART B , BAGNELL J A ,et al. Activity forecasting[C]// Proceedings of the 12th European conference on Computer Vision - Volume Part IV. Berlin:Springer, 2012: 201-214.
[4]	LEE N , CHOI W , VERNAZA P ,et al. Desire:distant future prediction in dynamic scenes with interacting agents[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 336-345.
[5]	YAMAGUCHI K , BERG A C , ORTIZ L E ,et al. Who are you with and where are you going?[C]// Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2011: 1345-1352.
[6]	HOCHREITER S , SCHMIDHUBER J . Long short-term memory[J]. Neural Computation, 1997,9(8): 1735-1780.
[7]	VEMULA A , MUELLING K , OH J . Social attention:modeling attention in human crowds[C]// 2018 IEEE International Conference on Robotics and Automation. Piscataway:IEEE Press, 2018: 1-7.
[8]	ALAHI A , GOEL K , RAMANATHAN V ,et al. Social LSTM:human trajectory prediction in crowded spaces[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 961-971.
[9]	GUPTA A , JOHNSON J,FEI-FEI L , et al . Social GAN:socially acceptable trajectories with generative adversarial networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2018: 2255-2264.
[10]	GOODFELLOW I , POUGET-ABADIE J , MIRZA M ,et al. Generative adversarial nets[C]// Advances in Neural Information Processing Systems. Cambridge:MIT Press, 2014: 2672-2680.
[11]	SADEGHIAN A , KOSARAJU V , SADEGHIAN A ,et al. SoPhie:an attentive GAN for predicting paths compliant to social and physical constraints[J]. ArXiv Preprint,ArXiv:1806.01482, 2018
[12]	VINEET K , AMIR S , ROBERTO M ,et al. Social-BiGAT:multimodal trajectory forecasting using bicycle-GAN and graph attention networks[J]. ArXiv Preprint ArXiv:1907.03395, 2019
[13]	PETAR V , GUILLEM C , ARANTXA C ,et al. Graph attention networks[J]. CoRR,abs/1710.10903, 2018
[14]	MNIH V , HEESS N , GRAVES A ,et al. Recurrent models of visual attention[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014: 2204-2212.
[15]	KINGMA D P , WELLING M . Auto-encoding variational Bayes[J]. ArXiv preprint,ArXiv:1312.6114, 2013
[16]	VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Advances in Neural Information Processing Systems. Cambridge:MIT Press, 2017: 5998-6008.
[17]	孙亚圣, 姜奇, 胡洁 ,等. 基于注意力机制的行人轨迹预测生成模型[J]. 计算机应用, 2019,39(3): 668-674.
	SUN Y S , JIANG Q , HU J ,et al. Attention mechanism based pedestrian trajectory prediction generation model[J]. Journal of Computer Applications, 2019,39(3): 668-674.
[18]	XU K , BA J , KIROS R ,et al. Show,attend and tell:neural image caption generation with visual attention[J]. ArXiv Preprint,ArXiv:1502.03044, 2015
[19]	MIRZA M , OSINDERO S . Conditional generative adversarial nets[J]. ArXiv Preprint,ArXiv:1411.1784, 2014
[20]	XU Q , HUANG G , YUAN Y ,et al. An empirical study on evaluation metrics of generative adversarial networks[J]. ArXiv Preprint,ArXiv:1806.07755, 2018
[21]	PELLEGRINI S , ESS A , VAN GOOL L . Improving data association by joint modeling of pedestrian trajectories and groupings[C]// European Conference on Computer Vision. Berlin:Springer, 2010: 452-465.
[22]	LERNER A , CHRYSANTHOU Y , LISCHINSKI D . Crowds by example[J]. Computer Graphics Forum, 2007,26(3): 655-664.
[23]	ABDUALLAH M , KUN Q , MOHAMED E ,et al. Social-STGCNN:a social spatio-temporal graph convolutional neural network for human trajectory prediction[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020.
[24]	HE Z , CHOW C , ZHANG J . STCNN:a spatio-temporal convolutional neural network for long-term traffic prediction[C]// 2019 20th IEEE International Conference on Mobile Data Management. Piscataway:IEEE Press, 2019: 226-233.

Metrics

Recommended 0

No Suggested Reading articles found!

数据集			ADE/m				FDE/m
数据集	Gumbel Pooling	Max Pooling	Average Pooling	Random Pooling	Gumbel Pooling	Max Pooling	Average Pooling	Random Pooling
ETH	0.58	0.55	0.64	0.60	1.17	1.05	1.24	1.14
Hotel	0.31	0.45	0.29	0.34	0.62	0.86	0.55	0.65
Univ	0.34	0.52	0.37	0.37	0.69	0.93	0.75	0.74
Zara1	0.21	0.54	0.22	0.21	0.45	0.98	0.42	0.41
Zara2	0.21	0.42	0.22	0.21	0.43	0.72	0.44	0.42
平均	0.33	0.496	0.348	0.346	0.672	0.908	0.68	0.672

预测时间	数据集				评价基准ADE/FDE
预测时间	数据集	Linear	LSTM	S-LSTM	S-GAN	Sophie	Social-BiGAT	SA-GAN
	ETH	0.84/1.60	0.70/1.45	0.73/1.48	0.67/1.47	—	—	0.58/1.17
	Hotel	0.35/0.60	0.55/1.17	0.49/1.01	0.52/1.05	—	—	0.31/0.62
t_pred=8	Univ	0.56/1.01	0.36/0.77	0.41/0.84	0.34/0.75	—	—	0.34/0.69
	Zara1	0.41/0.74	0.25/0.74	0.27/0.56	0.26/0.45	—	—	0.21/0.45
	Zara2	0.53/0.95	0.31/0.65	0.33/0.79	0.29/0.58	—	—	0.21/0.43
	平均	0.53/0.98	0.43/0.91	0.45/0.91	0.42/0.86	—	—	0.33/0.67
	ETH	1.33/2.94	1.09/2.41	1.09/2.35	0.92/1.73	0.70/1.43	0.69/1.29	0.72/1.28
	Hotel	0.39/0.72	0.86/1.91	0.79/1.76	0.67/1.37	0.76/1.67	0.49/1.01	0.50/1.01
t_pred=12	Univ	0.82/1.59	0.61/1.31	0.67/1.40	0.76/1.52	0.54/1.24	0.55/1.32	0.58/1.19
	Zara1	0.62/1.21	0.41/0.88	0.47/1.00	0.35/0.68	0.30/0.63	0.30/0.62	0.42/0.83
	Zara2	0.77/1.48	0.52/1.11	0.56/1.17	0.42/0.84	0.38/0.78	0.36/0.75	0.39/0.85
	平均	0.79/1.59	0.70/1.52	0.72/1.54	0.62/1.23	0.54/1.15	0.48/1.00	0.52/1.03

预测模型	预测8帧时间/min	预测12帧时间/min
LSTM	0.029	0.042
S-LSTM	0.504	0.825
S-GAN	0.202	0.341
SA-GAN	0.163	0.282

Research on pedestrian trajectory prediction method based on social attention mechanism

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 24

Related Articles 15

Metrics

Recommended 0

[1]	Jiale ZHANG, Chengcheng ZHU, Xiaobing SUN, Bing CHEN. Membership inference attack and defense method in federated learning based on GAN [J]. Journal on Communications, 2023, 44(5): 193-205.
[2]	Xin SUN, Guifu ZHANG, Hongyan XING, Wang Zenghui. Research on intrusion detection for maritime meteorological sensor network based on balancing generative adversarial network [J]. Journal on Communications, 2023, 44(4): 124-136.
[3]	Lingtao TANG, Di WANG, Shengyun LIU. Data augmentation scheme for federated learning with non-IID data [J]. Journal on Communications, 2023, 44(1): 164-176.
[4]	Yanhua LIU, Jiaqi LI, Zhengui OU, Xiaoling GAO, Ximeng LIU, Weizhi MENG, Baoxu LIU. Adversarial training driven malicious code detection enhancement method [J]. Journal on Communications, 2022, 43(9): 169-180.
[5]	Yanwen WANG, Weimin LEI, Wei ZHANG, Huan MENG, Xinyi CHEN, Wenhui YE, Qingyang JING. Survey on video image reconstruction method based on generative model [J]. Journal on Communications, 2022, 43(9): 194-208.
[6]	Xueyuan DUAN, Yu FU, Kun WANG. Multi-dimensional time series anomaly detection method based on VAE-WGAN [J]. Journal on Communications, 2022, 43(3): 1-13.
[7]	Hailin FENG, Xiao ZHANG, Tongcun LIU. Recommendation model combining review’s feature and rating graph convolutional representation [J]. Journal on Communications, 2022, 43(3): 164-171.
[8]	Xiayu XIANG, Jiahui WANG, Zirui WANG, Shaoming DUAN, Hezhong PAN, Rongfei ZHUANG, Peiyi HAN, Chuanyi LIU. Generate medical synthetic data based on generative adversarial network [J]. Journal on Communications, 2022, 43(3): 211-224.
[9]	Meiyu ZHONG, Peiliang WU, Yan DOU, Yi LIU, Lingfu KONG. Chinese semantic and phonological information-based text proofreading model for speech recognition [J]. Journal on Communications, 2022, 43(11): 65-79.
[10]	Zhuo CHEN, Miao ZHU, Junwei DU. Multi-view graph neural network for fraud detection algorithm [J]. Journal on Communications, 2022, 43(11): 225-232.
[11]	Yanhui LU, Han LIU, Hang LI, Guangxu ZHU. Time series generation model based on multi-discriminator generative adversarial network [J]. Journal on Communications, 2022, 43(10): 167-176.
[12]	Wei LIU, Cheng CHEN, Rui JIANG, Tao LU. Four-path unsupervised learning-based image defogging network [J]. Journal on Communications, 2022, 43(10): 210-222.
[13]	Hongyan WANG, Hai YUAN. Action recognition method based on fusion of skeleton and apparent features [J]. Journal on Communications, 2022, 43(1): 138-148.
[14]	Zhili ZHOU, Meimin WANG, Gaobo YANG, Jianyu ZHU, Xingming SUN. Generative steganography method based on auto-generation of contours [J]. Journal on Communications, 2021, 42(9): 144-154.
[15]	Xiaojuan ZHAO, Yan JIA, Aiping LI, Kai CHEN. Research on link prediction model based on hierarchical attention mechanism [J]. Journal on Communications, 2021, 42(3): 36-44.