Pedestrian-crossing intention-recognition based on dual-stream adaptive graph-convolutional neural-network

doi:10.3969/j.issn.1674-8484.2022.02.013

Abstract

Abstract:

A recognition method was proposed to judge pedestrians’ intention to cross street for autonomous vehicles on urban roads. The method utilized a two-stream, spatiotemporally adaptive graph-convolutional neural-network (named 2s-AGCN) with linking the dynamics of pedestrian skeletons and pedestrian crossing intention; Added the adaptive graph convolutional neural-network (AGCN) structure based on the action recognition of the spatiotemporal graph convolutional neural network (ST-GCN); A dual-stream neural-network was designed in terms of the length and direction of the bones for fusing the Softmax scores output by the two networks to predict pedestrian crossing intention. Simulation experiments were carried out based on the Joint Attention in Autonomous Driving public Dataset (JAAD). The results shown that the accuracy of this 2s-AGCN method reaches 89.36%, which is 3.36% higher than the accuracy of the ST-GCN. Therefore, the recognition accuracy of this method is high.

Key words: autonomous vehicles, driving safety, pedestrian crossing intention, graph convolution neural-network (GCN)

CLC Number:

U463.6

HU Yuanzhi, JIANG Tao, LIU Xi, SHI Youning. Pedestrian-crossing intention-recognition based on dual-stream adaptive graph-convolutional neural-network[J]. Journal of Automotive Safety and Energy, 2022, 13(2): 325-332.

Figures/Tables 12

References 25

[1]	Gesnouin J, Pechberti S, Bresson G, et al. Predicting intentions of pedestrians from 2D skeletal pose sequences with a representation-focused multi-branch deep learning network[J]. Algorithms, 2020, 13(12): 1-8. doi: 10.3390/a13010001 URL
[2]	Donahue J, Hendricks L A, Guadarrama S, et al. Long-term recurrent convolutional networks for visual recognition and description[C]// 2015 IEEE Conf Comp Vis Patt Recog (CVPR), UC Berkeley, USA. 2015: 2625-2634.
[3]	Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3D Convolutional networks[C]// 2015 IEEE Int’l Conf Comp Vis (ICCV), Hanover, NH, USA. 2015: 4489-4497.
[4]	Varol G, Laptev I, Schmid C. Long-term temporal convolutions for action recognition[J]. IEEE Trans Pattern Ana Mach Intell, 2018, 40(6): 1510-1517.
[5]	WU Chaoyuan, Zaheer M, HU Hexiang, et al. Compressed video action recognition[C]// Proc IEEE Conf Compu Vis Patt Recog (CVPR), Austin, TX, USA. 2018: 6026-6035.
[6]	ZHANG Bowen, WANG Liming, WANG Zhe, et al. Real-time action recognition with enhanced motion vector CNNs[C]// Proc IEEE Conf Computer Vision Pattern Recog, Las Vegas, NV, USA. 2016 : 2718-2726.
[7]	Sevilla-lara L, LIAO Yiyi, Guney F, et al. On the Integration of optical flow and action recognition[C]// Proc German Conf Pattern Recog, Stuttgart, Germany. 2018: 281-297.
[8]	Pop D O, Rogozan A, Chatelain C, et al. Multi-task deep learning for pedestrian detection, action recognition and time to cross prediction[J]. IEEE Access, 2019, 7: 149318-149327. doi: 10.1109/ACCESS.2019.2944792 URL
[9]	Vemulapalli R, Arrate F, Chellappa R. Human action recognition by representing 3D skeletons as points in a lie group, 2014 IEEE Conf Comp Vis Pat Recog (CVPR), Columbus, OH, USA. 2014: 558-595.
[10]	DU Yong, WEI Wang, LIANG Wang. Hierarchical recurrent neural network for skeleton based action recognition[C]// Proc IEEE Conf Comp Vis Patt Recog, Boston, MA, USA. 2015: 1110-1118.
[11]	LIU Jun, Shahroudy A, DONG Xu, et al. Spatio-temporal LSTM with trust gates for 3D human action recognition[J]. IEEE Trans Pattern Ana Mach Intel, 2018, 40(12): 3007-3021.
[12]	YAN Sijie, XIONG Yuanjun, LIN Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]// Proc of AAAI (Asso Adva Artif Intell) Conf Artif Intell New Orleans, LA, USA. 2018: 956-965.
[13]	Shahroudy A, LIU Jun, Ng T T, et al. NTU RGB+D: A large scale dataset for 3D human activity analysis[C]// IEEE Computer Soc, Las Vegas, NV, USA. 2016 : 1010-1019.
[14]	SONG Sijie, LAN Cuiling, XING Junliang, et al. An end-to-end spatio-temporal attention model for human action recognition from skeleton data[C]// AAAI 2017: Proc 31th AAAI Conf Artif Intelle, San Francisco, CA, USA. 2017 : 4263-4270.
[15]	CAO Zhe, Simon T, WEI Shihen, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]// 2017 IEEE Conf Compu Vis Patt Recog (CVPR), Pittsburgh, PA, USA. 2017: 1302-1310.
[16]	FANG Zhijie, López A. Is the Pedestrian going to Cross? Answering by 2D pose estimation[C]// 2018 IEEE Intel Vehi Symp (IV), Changshu, China. 2018: 1271-1276.
[17]	Quintero R, Parra I, Lorenzo J, et al. Pedestrian intention recognition by means of a Hidden Markov Model and body language[C]// 2017 IEEE 20th Int’l Conf Intell Transp Syst (ITSC), Alcala de Genares, Spain. 2017: 1-7.
[18]	DONG Xiaowen, Thanou D, Rabbat M, et al. Learning graphs from data: A signal representation perspective[J]. IEEE Signal Proce Maga, 36(3): 44-63.
[19]	WANG Xiaolong, Girshick R, Gupta A, et al. Non-local neural networks[C]// CVPR 2018: Proc 2018 IEEE/CVF Conf Compu Vis Patt Recog, Salt Lake, UT USA. 2018: 7794-7803.
[20]	Rasouli A, Kotseruba I, Tsotsos J K. Are they going to cross? A benchmark dataset and baseline for pedestrian crosswalk behavior[C]// IEEE Int’l Conf Computer Vis Workshop, Redondo Beach, CA, USA. 2017: 206-213.
[21]	Varytimidis D, Alonso-fernandez F, Duran B, et al. Action and intention recognition of pedestrians in urban traffic[C]// 2018 14th Int’l Conf Signal-Image Tech Internet-Based Syst (SITIS), Gran Canaria, Spain, 2018: 26-29.
[22]	Marginean A, Brehar R, Negru M. Understanding pedestrian behaviour with pose estimation and recurrent networks[C]// 2019 6th Int’l Symp Elect Electro Engi (ISEEE), Galati, Romania. 2019: 1-6.
[23]	Gujjar P, Vaughan R. Classifying pedestrian actions in advance using predicted video of urban driving scenes[C]// Proc 2019 Int’l Conf Robo Automa (ICRA), Montreal, QC, Canada. 2019: 2097-2103.
[24]	Saleh K, Hossny M, Nahavandi S. Real-time intent prediction of pedestrians for autonomous ground vehicles via spatio-temporal DenseNet[C]// Int’l Conf Robo Automa, Deakin, Australia. 2019: 9704-9710.
[25]	Chaabane M, Trabelsi A, Blanchard N, et al. Looking ahead: Anticipating pedestrians crossing with future frames prediction[C]// 2020 IEEE Winter Conf Appl Comp Vis (WACV), Fort Collins, CO, USA. 2020: 2286-2295.

方法	文献	准确率 / %
Alexnet+Context	[20]	63.00
Alexnet+SVM	[21]	74.00
Alphapose+LSTM	[22]	78.00
Res-EnDec	[23]	81.00
ST-DenseNet	[24]	84.00
auto-encoder+Prediction	[25]	86.00
Openpose+Keypoint	[16]	88.00
Openpose+ST-GCN	本文	86.00
Openpose+2s-AGCN	本文	89.36

方法	文献	准确率 / %
Alexnet+Context	[20]	63.00
Alexnet+SVM	[21]	74.00
Alphapose+LSTM	[22]	78.00
Res-EnDec	[23]	81.00
ST-DenseNet	[24]	84.00
auto-encoder+Prediction	[25]	86.00
Openpose+Keypoint	[16]	88.00
Openpose+ST-GCN	本文	86.00
Openpose+2s-AGCN	本文	89.36