车载视角下基于视觉信息的前车行为识别

doi:10.3969/j.issn.1674-8484.2023.06.007

汽车安全与节能学报 ›› 2023, Vol. 14 ›› Issue (6): 707-714.DOI: 10.3969/j.issn.1674-8484.2023.06.007

车载视角下基于视觉信息的前车行为识别

刘延伟¹(), 黄志明¹^,², 高博麟²^,^*(), 钟薇³, 陈嘉星², 刘家熙²

1.广东工业大学机电工程学院，广州510006，中国
2.清华大学，汽车安全与节能国家重点实验室，北京 100084，中国
3.国家智能网联汽车创新中心，北京 100084，中国

收稿日期:2023-03-09 修回日期:2023-09-07 出版日期:2023-12-31 发布日期:2023-12-29
通讯作者: * 高博麟，副研究员。E-mail：gaobolin@tsinghua.edu.cn。
作者简介:刘延伟（1985—），男（汉），甘肃，副教授。E-mail：ywliu@gdut.edu.cn。
基金资助:
国家自然科学基金(52172389);广东自然科学基金(2022A1515012080)

Recognition of front vehicle behavior based on visual information from vehicle perspective

LIU Yanwei¹(), HUANG Zhiming¹^,², GAO Bolin²^,^*(), ZHONG Wei³, CHEN Jiaxing², LIU Jiaxi²

1. School of Electro-Mechanical Engineering, Guangdong University of Technology, Guangzhou 510006, China
2. State Key Laboratory of Automotive Safety and Energy, Tsinghua University, Beijing 100084, China
3. National Innovation Center of Intelligent and Connected Vehicles, Beijing, 100084, China

Received:2023-03-09 Revised:2023-09-07 Online:2023-12-31 Published:2023-12-29

摘要/Abstract

摘要：

不同于传统车辆行为识别方法大多基于鸟瞰视角下的历史轨迹信息，该文提出了一种用于自动驾驶汽车的车载视角下基于视觉信息的前车行为识别方法。针对缺乏车辆行为数据集的问题，提出了基于车载视频信息的车辆行为标注方法，构建了车载视角下的行为识别数据集。设计了一种基于SlowFast网络为主体的车辆行为识别算法，设计了焦点损失函数，并引入非局部操作模块来替换原有的交叉熵损失函数。结果表明：相较于原SlowFast模型，新模型的总体准确率提高了20.56%，实现了对视频中前方多台车辆的行为识别。

关键词: 自动驾驶汽车, 车端感知, 前车行为识别, SlowFast网络, 车载视角

Abstract:

Unlike traditional vehicle behavior recognition methods, which are mostly based on historical track information from a bird's eye view, this paper proposed a new approach for autonomous vehicle behavior recognition based on visual information. A vehicle behavior labeling method based on vehicle-mounted video information was proposed, and a behavior recognition data set was constructed from vehicle-mounted perspective. A vehicle behavior recognition algorithm based on SlowFast network was designed; A Focal Loss function was designed; And a Non-local operation module was introduced. The results show that compared with the original SlowFast model, the overall accuracy of the new model is improved by 20.56%, and the behavior recognition of multiple vehicles in front of the video is realized.

Key words: autonomous vehicles, on-board perception, front vehicle behavior recognition, SlowFast networks, vehicle perspective

中图分类号:

U471

刘延伟, 黄志明, 高博麟, 钟薇, 陈嘉星, 刘家熙. 车载视角下基于视觉信息的前车行为识别[J]. 汽车安全与节能学报, 2023, 14(6): 707-714.

LIU Yanwei, HUANG Zhiming, GAO Bolin, ZHONG Wei, CHEN Jiaxing, LIU Jiaxi. Recognition of front vehicle behavior based on visual information from vehicle perspective[J]. Journal of Automotive Safety and Energy, 2023, 14(6): 707-714.

图/表 10

参考文献 17

[1]	王相海, 丛志环, 方玲玲, 等. 基于 HMM 的车辆行驶状态实时判别方法研究[J]. 自动化学报, 2013, 39(12): 2131-2142.
	WANG Xianghai, CONG Zhihuai, FANG lingling, et al. Determination of real-time vehicle driving status using HMM[J]. Acta Automatica Sinica, 2013, 39(12): 2131-2142. (in Chinese) doi: 10.3724/SP.J.1004.2013.02131 URL
[2]	XU Donghao, HE Xu, ZHAO Huijing, et al. Ego-centric traffic behavior understanding through multi-level vehicle trajectory analysis[C]//2017 IEEE Int’l Conf Robo Automa (ICRA), 2017: 211-218.
[3]	LIU Zhi, WU Xuegang, NI Jie, et al. Driving intention recognition based on HMM and SVM cascade algorithm[J]. Autom Engineering, 2018, 40(7): 858-864.
[4]	黄慧玲, 杨明, 王春香, 等. 基于前方车辆行为识别的碰撞预警系统[J]. 华中科技大学学报(自然科学版), 2015, 43(S1): 117-121.
	HUANG Huiling, YANG Ming, WANG Chunxiang, et al. Collision warning system based on forward vehicle behavior recognition[J]. J Huazhong Univ Sci Tech (Nat Sci Edit), 2015, 43(S1) : 117-121. (in Chinese)
[5]	党彤. 基于贝叶斯网络的车辆变道行为分析[D]. 西安: 西安理工大学, 2018.
	DANG Tong. Analysis of vehicle lane change behavior based on Bayesian network[D]. Xi'an: Xi'an University of Technology, 2018. (in Chinese)
[6]	卫星, 乐越, 韩江洪, 等. 基于长短期记忆的车辆行为动态识别网络[J]. 计算机应用, 2019, 39(7): 1894-1898. doi: 10.11772/j.issn.1001-9081.2018122448
	WEI Xing, LE Yue, HAN Jianghong, et al. Vehicle behavior dynamic recognition network based on long short-term memory[J]. J Comp Appl, 2019, 39(7): 1894-1898. (in Chinese)
[7]	季学武, 费聪, 何祥坤, 等. 基于LSTM网络的驾驶意图识别及车辆轨迹预测[J]. 中国公路学报, 2019, 32(6): 34-42. doi: 10.19721/j.cnki.1001-7372.2019.06.003
	JI Xuewu, FEI Cong, HE Xiangkun, et al. Driving intention recognition and vehicle trajectory prediction based on LSTM network[J]. Chin J Highw Transport, 2019, 32(6): 34-42. (in Chinese) doi: 10.19721/j.cnki.1001-7372.2019.06.003
[8]	蔡英凤, 邰康盛, 王海, 等. 无人驾驶汽车周边车辆行为识别算法研究[J]. 汽车工程, 2020, 42(11): 1464-1472. doi: 10.19562/j.chinasae.qcgc.2020.11.003
	Cai Yingfeng, Tai Kangsheng, Wang Hai et al. Research on behavior recognition algorithm of surrounding vehicles for driverless ca[J]. Autom Engineering, 2020, 42(11): 1464-1472. (in Chinese)
[9]	MOU Luntian, XIE Haitao, MAO Shasha, et al. Vision-based vehicle behaviour analysis: A structured learning approach via convolutional neural networks[J]. IET Intel Transport Syst, 2020, 14(7): 792-801.
[10]	Krajewski R, Bock J, Kloeker L, et al. The highd dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems[C]// 2018 21st Int’l Conf Intel Transp Syst (ITSC), 2018: 2118-2125.
[11]	Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? the KITTI vision benchmark suite[C]// 2012 IEEE Conf Comp Visi Patt Recog, 2012: 3354-3361.
[12]	YU Fisher, CHEN Haofeng, WANG Xin, et al. BDD100K: A diverse driving dataset for heterogeneous multitask learning[C]//Proc IEEE/CVF Conf Comp Visi Patt Recog, 2020: 2636-2645.
[13]	Feichtenhofer C, FAN Haoqi, Malik J, et al. Slowfast networks for video recognition[C]// Proc IEEE/CVF Int’l Conf Comp Visi, 2019: 6202-6211.
[14]	Carreira J, Zisserman A. Quo vadis, action recognition? A new model and the kinetics dataset[C]// Proc IEEE Conf Comp Visi Patt Recog, 2017: 6299-6308.
[15]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]// Pro IEEE Conf Comp Visi Patt Recog, 2016: 770-778.
[16]	LIN Tsung-Yi, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]// Proc IEEE Int’l Conf Comp Visi, 2017: 2980-2988.
[17]	WANG Xiaolong, Girshick R, Gupta A, et al. Non-local neural networks[C]// Proc IEEE Conf Comp Visi Patt Recog, 2018: 7794-7803.

帧序号	X1	Y1	X2	Y2	行为标签
31	0.397	0.460	0.466	0.555	2
41	0.462	0.480	0.503	0.536	2
51	0.478	0.477	0.512	0.527	2
61	0.556	0.479	0.646	0.592	2
71	0.534	0.479	0.606	0.570	2
81	0.515	0.477	0.574	0.557	1
91	0.504	0.479	0.554	0.546	1
101	0.520	0.479	0.583	0.561	1
111	0.515	0.477	0.574	0.557	1

帧序号	X1	Y1	X2	Y2	行为标签
31	0.397	0.460	0.466	0.555	2
41	0.462	0.480	0.503	0.536	2
51	0.478	0.477	0.512	0.527	2
61	0.556	0.479	0.646	0.592	2
71	0.534	0.479	0.606	0.570	2
81	0.515	0.477	0.574	0.557	1
91	0.504	0.479	0.554	0.546	1
101	0.520	0.479	0.583	0.561	1
111	0.515	0.477	0.574	0.557	1

阶段	慢速路径	快速路径	输出大小：T×S²
视频Clips	-	-	64×224²
Data layer	Stride 16, 1²	Stride 16, 1²	Slow: 4×224² Fast: 32×224²
Conv1	1×7², 64 Stride 1, 2²	5×7², 8 Stride 2, 1²	Slow: 4×112² Fast: 32×112²
Pool1	1×3², max Stride 1, 2²	1×3², max Stride 1, 2²	Slow: 4×56² Fast: 32×56²
Res2			Slow: 4×56² Fast: 32×56²
Res3			Slow: 4×28² Fast: 32×28²
Res4			Slow: 4×14² Fast: 32×14²
Res5			Slow: 4×7² Fast: 32×7²
	平均池化，全连接层		Classes

阶段	慢速路径	快速路径	输出大小：T×S²
视频Clips	-	-	64×224²
Data layer	Stride 16, 1²	Stride 16, 1²	Slow: 4×224² Fast: 32×224²
Conv1	1×7², 64 Stride 1, 2²	5×7², 8 Stride 2, 1²	Slow: 4×112² Fast: 32×112²
Pool1	1×3², max Stride 1, 2²	1×3², max Stride 1, 2²	Slow: 4×56² Fast: 32×56²
Res2			Slow: 4×56² Fast: 32×56²
Res3			Slow: 4×28² Fast: 32×28²
Res4			Slow: 4×14² Fast: 32×14²
Res5			Slow: 4×7² Fast: 32×7²
	平均池化，全连接层		Classes

方案	模型	精确率 / %					召回率 / %					总体准确率/ %
		变道		转弯		横行穿越	变道		转弯		横行穿越
		左	右	左	右	横行穿越	左	右	左	右	横行穿越
1	SlowFast	66.20	43.10	34.59	36.05	47.83	47.96	59.53	23.23	24.41	81.76	47.56
2	SlowFast + Focal Loss	72.57	64.58	44.55	52.48	53.16	71.09	57.67	47.47	41.73	68.24	59.16
3	SlowFast + Non-local	73.70	70.87	43.84	63.64	63.45	72.45	67.91	61.11	33.07	62.16	62.52
4	SlowFast + Focal Loss + Non-local	78.74	77.44	47.19	70.73	70.80	80.61	70.23	63.64	45.67	65.54	68.12

车载视角下基于视觉信息的前车行为识别

Recognition of front vehicle behavior based on visual information from vehicle perspective

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 17

相关文章 9

编辑推荐

Metrics

本文评价

期刊信息

在线期刊

作者中心

审稿中心

联系我们

[1]	黄新朝, 张毅. 基于百度地图API的纯电动汽车未来行驶能耗预测[J]. 汽车安全与节能学报, 2023, 14(6): 715-722.
[2]	杨金松, 赵德宗, 江菁晶, 蓝江林, 李亮. 异构智能网联车辆编队的双阶段节能驾驶控制策略(英文)[J]. 汽车安全与节能学报, 2022, 13(4): 676-685.
[3]	李文礼, 邱凡珂, 廖达明, 任勇鹏, 易帆. 基于深度强化学习的高速公路换道跟踪控制模型[J]. 汽车安全与节能学报, 2022, 13(4): 750-759.
[4]	张平, 陈一凡, 江书真, 韩毅. 高速公路上自动超车过程的轨迹规划与跟踪控制[J]. 汽车安全与节能学报, 2022, 13(3): 463-472.
[5]	武一民, 郑凯元, 高博麟, 陈明, 王义锋. 基于自适应扩展Kalman滤波的路侧多传感器融合[J]. 汽车安全与节能学报, 2021, 12(4): 522-527.
[6]	李磊, 李军, 张世义. 搭载改进滑模控制的自动驾驶汽车轨迹跟踪控制[J]. 汽车安全与节能学报, 2020, 11(4): 503-510.
[7]	韩大双, 马志雄, 朱西产, 曾宇凡. 用于自动驾驶汽车的汽车—骑车人事故场景分析 [J]. 汽车安全与节能学报, 2020, 11(2): 220-226.
[8]	谢辉，高斌，熊硕，王悦. 结构化道路中动态车辆的轨迹预测[J]. JASE, 2019, 10(4): 413-422.
[9]	李学鋆. 基于UTMD 的汽车自动驾驶的路径规划寻优算法[J]. JASE, 2018, 9(4): 449-455.