基于多源传感器与导航地图的端到端自动驾驶方法

doi:10.3969/j.issn.1674-8484.2022.04.015

汽车安全与节能学报 ›› 2022, Vol. 13 ›› Issue (4): 738-749.DOI: 10.3969/j.issn.1674-8484.2022.04.015

基于多源传感器与导航地图的端到端自动驾驶方法

朱波¹^,²(), 张纪伟¹(), 谈东奎¹^,²^,^*(), 胡旭东¹

1.合肥工业大学汽车工程技术研究院，合肥市 230009，中国
2.合肥工业大学智能制造技术研究院，合肥市 230051，中国

收稿日期:2022-05-21 修回日期:2022-07-25 出版日期:2022-12-31 发布日期:2023-01-01
通讯作者: 谈东奎
作者简介:*谈东奎，助理研究员。E-mail：tandongkui@126.com。
朱波(1980—)，男(汉)，湖北，副研究员。E-mail: zhubo@hfut.edu.cn。
张纪伟(1993—)，男(汉)，河南，硕士研究生。E-mail: z18336345753@163.com。
基金资助:
安徽省科技重大专项(201903a05020016);安徽省重点研究与开发计划(202004b11020002);安徽省自然科学基金(2208085QE153)

End-to-end autonomous driving method based on multi-source sensor and navigation map

ZHU Bo¹^,²(), ZHANG Jiwei¹(), TAN Dongkui¹^,²^,^*(), HU Xudong¹

1. Automotive Research Institute, Hefei University of Technology, Hefei 230009, China
2. Intelligent Manufacturing Institute, Hefei University of Technology, Hefei 230009, China

Received:2022-05-21 Revised:2022-07-25 Online:2022-12-31 Published:2023-01-01
Contact: TAN Dongkui

摘要/Abstract

摘要：

提出了一种基于多源传感器与导航地图的多端输入-单端输出(端到端)自动驾驶决策控制模型，以弥补现有端到端自动驾驶方法中基于深度神经网络(DNN)的PilotNet模型在主动避障行驶和交叉路口通行方面的不足。该模型的传感器数据输入端包括：单目前视摄像头、360( ° )多线激光雷达(LiDAR)所得二维俯视图、精准定位的局部导航地图等3部分；车辆控制命令输出端为方向盘转向角。进行了多工况仿真和实车试验。结果表明：与PilotNet模型相比，该模型的方向盘转向角均方根误差(RMSE)值下降了37%；因而，该模型具备主动避障和交叉路口通行的能力。

关键词: 自动驾驶车辆, 端到端自动驾驶, 神经网络, 摄像头, 导航地图, 激光雷达(LiDAR)

Abstract:

This paper proposed a multiple-input single-output (end-to-end) autonomous driving decision control model based on multi-source sensors and navigation map to make up for the shortcomings of existing end-to-end autonomous driving methods in active obstacle avoidance and passing through intersection by using the PilotNet model based on Deep Neural Network (DNN). The sensor data in model’s input end consisted of three parts: a monocular front view camera, a 2-D top view obtained by 360° multi-layer Light-Laser Detection and Ranging (LiDAR), and a local navigation map based on accurate positioning. The model’s output end generated the steering wheel angle, which was a vehicle control command. Some multi-condition simulations and real vehicle tests were conducted. The results show that the root mean square error (RMSE) of steering wheel angle decreases by 37% compared with that by using the PilotNet model. Therefore, the model has the ability of active obstacle avoidance and intersection passing.

Key words: autonomous vehicles, end-to-end autonomous driving, neural networks, camera, navigation maps, light-laser detection and ranging (LiDAR)

中图分类号:

U467.1+3

朱波, 张纪伟, 谈东奎, 胡旭东. 基于多源传感器与导航地图的端到端自动驾驶方法[J]. 汽车安全与节能学报, 2022, 13(4): 738-749.

ZHU Bo, ZHANG Jiwei, TAN Dongkui, HU Xudong. End-to-end autonomous driving method based on multi-source sensor and navigation map[J]. Journal of Automotive Safety and Energy, 2022, 13(4): 738-749.

图/表 26

图1 3种端到端决策模型架构

整车装备质量，m	1.532 t
前轴到车辆质心的距离, l_f	1.21 m
后轴到车辆质心的距离, l_r	1.5 m
质心高度, h_cog	0.5 m
车辆的宽度, w_v	2.07 m
车辆的长度 l	4.63 m

表1 车辆主要参数

图2 LiDAR坐标系和图像坐标系

图3 16线激光雷达俯视图

图4 局部导航地图截取流程

图5 局部导航地图旋转前后的对比

图6 车辆初始化偏离车道中心

图7 Pure Pursuit算法跟踪模型

图8 Dataset-C转向角统计

图9 Model-C模型训练过程示意图

图10 Model-C模型训练Loss变化曲线

网络模型	RMSE / (°)
PilotNet	5.67
Model-A	2.81
Model-B	3.23
Model-C	3.57

表2 不同模型的性能对比

图11 车道保持验证场景

图12 基础模型/Model-A/PilotNet转向角预测

图13 Model-A/B/C转向角预测

图14 避障验证场景图

图15 避障场景下Model-A/B/C的车辆轨迹

图16 车辆障碍物场景下基于Dataset-B的Model-A/B的车辆轨迹

图17 交叉路口验证场景

图18 全局导航路径nav-1和nav-2

图19 Model-B/C预测的车辆行驶轨迹对比

模型	输入数据	t / ms	车道保持功能	主动避障行驶	通过交叉路口
PilotNet	摄像头	3.9	√	×	×
Model-A	摄像头	5.6	√	×	×
Model-B	摄像头 + LiDAR	9.8	√	√	×
Model-C	摄像头 + LiDAR + 导航地图	12.7	√	√	√

表3 不同模型间的功能表现对比

图20 智能驾驶电动汽车试验平台

图21 园区道路俯视图及测试路线

图22 真实场景道路图像示例

图23 人类驾驶员与Model-C的转向角对比

参考文献 21

[1]	陈颖. 自动驾驶技术研究现状及发展趋势[J]. 能源技术与管理, 2021, 46(3): 34-37.
	CHEN Ying. Research status and development trend of automatic driving technology[J]. *Energy Tech Manag*, 2021, 46(3): 34-37. (in Chinese)
[2]	Broggi A, Cerri P, Debattisti S, et al. PROUD Public road urban driverless-car test[J]. *IEEE Trans Intell Transp Syst*, 2015, 16(6): 3508-3519.
[3]	Bojarski M, Yeres P, Choromanska A, et al. Explaining how a deep neural network trained with end-to-end learning steers a car[EB/OL]. (2017-04-25), https://arxiv.org/abs/1704.07911.
[4]	WANG Tinghan, LUO Yugong, LIU Jinxin, et al. End-to-end self-driving approach independent of irrelevant roadside objects with auto-encoder[J]. *IEEE Trans Intell Transp Syst*, 2020, 23(1): 641-650.
[5]	Pomerleau D A. Alvinn: An autonomous land vehicle in a neural network[C]// Adva Neural Info Proc Syst. San Francisco: Morgan Kaufmann Publishers Inc, 1989: 305-313.
[6]	LeCun Y, Muller U, Ben J, et al. Off-road obstacle avoidance through end-to-end learning[C]// Int’l Conf Neural Info Proc Syst, Vancouver, 2005: 1-8.
[7]	Bojarski M, Testa D, Dworakowski D, et al. End to end learning for self-driving cars[EB/OL]. (2016-04-25), http://arxiv.org/abs/1604.07316.
[8]	XU Huazhe, GAO Yang, YU Fisher, et al. End-to-end learning of driving models from large-scale video datasets[C]// Proc IEEE Conf Computer Vision Pattern Recog, IEEE, Honolulu, 2017: 2174-2182.
[9]	胡学敏, 童秀迟, 郭琳, 等. 基于深度视觉注意神经网络的端到端自动驾驶模型[J]. 计算机应用, 2020, 40(7): 1926-1931. doi: 10.11772/j.issn.1001-9081.2019112054
	HU Xuemin, TONG Xiuchi, GUO Lin, et al. End-to-end autonomous driving model based on deep visual attention neural network[J]. *Computer Appl*, 2020, 40(7): 1926-1931. (in Chinese)
[10]	Codevilla F, Mller M, Lpez A, et al. End-to-end driving via conditional imitation learning[C]// 2018 IEEE Int’l Conf Robo Automation (ICRA). Brisbane, 2018: 4693-4700.
[11]	Sallab A, Abdou M, Perot E, et al. Deep reinforcement learning framework for autonomous driving[J]. *Electronic Imaging*, 2017(19): 70-76.
[12]	CHEN Siyuan, WANG Meiling, SONG Wenjie, et al. Stabilization approaches for reinforcement learning-based end-to-end autonomous driving[J]. *IEEE Trans Vehi Tech*, 2020, 69(5): 4740-4750.
[13]	SONG Wenjie, LIU Shixian, LI Yujun, et al. Smooth actor-critic algorithm for end-to-end autonomous driving[C]// 2020 Ame Control Conf, ACC, Colorado: 2020: 3242-3248.
[14]	Abdou M, Kamal H, El-Tantawy S, et al. End-to-end deep conditional imitation learning for autonomous driving[C]// 2019 31st Int’l Conf Micro- Electronics (ICM), IEEE, Beijing, 2020: 346-350.
[15]	卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述[J]. 数据采集与处理, 2016, 31(1): 1-17.
	LU Hongtao, ZHANG Qinchuan. A review of the application of deep convolutional neural networks in computer vision[J]. *Data Acqu Proce*, 2016, 31(1): 1-17. (in Chinese)
[16]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10), https://arxiv.org/abs/1409.1556.
[17]	Toth L. Phone recognition with deep sparse rectifier neural networks[C]// IEEE Acoustics Speech Signal Proc (ICASSP), IEEE, California: 2013: 6985-6989.
[18]	CHEN Zhilu, HUANG Xinming. End-to-end learning for lane keeping of self-driving cars[C]// 2017 IEEE Intell Vehi Symp (IV). IEEE, Los Angeles, 2017: 1856-1860.
[19]	Jhung J, Bae I, Moon J, et al. End-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles[C]// 2018 IEEE Intell Vehi Symp (IV). IEEE, Changshu, 2018: 617-622.
[20]	Mihalea A, Samoilescu R, Nica A, et al. End-to-end models for self-driving cars on UPB campus roads[C]// 2019 IEEE 15th Int’l Conf Intell Computer Commu Proc (ICCP). IEEE, Tokyo, 2019: 35-40.
[21]	YANG Zhengyuan, ZHANG Yixuan, YU Jerry, et al. End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perceptions[C]// 2018 24th Int’l Conf Pattern Recog (ICPR). IEEE, Beijing, 2018: 2289-2294.

基于多源传感器与导航地图的端到端自动驾驶方法

End-to-end autonomous driving method based on multi-source sensor and navigation map

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 26

参考文献 21

相关文章 15

编辑推荐

Metrics

本文评价

期刊信息

在线期刊

作者中心

审稿中心

联系我们

[1]	文家燕, 邹海峰, 钟薇, 高博麟, 卢彦博. 融合人车博弈论的车路云一体化系统车速规划方法[J]. 汽车安全与节能学报, 2026, 17(2): 261-269.
[2]	韩东明, 程思哲, 王金湘, 刘亚辉, 殷国栋. 混合交通下智能网联车辆预见性协同自适应巡航控制[J]. 汽车安全与节能学报, 2026, 17(1): 130-139.
[3]	付雪青, 陈闯, 周道卿, 张岩, 曹晓琳, 李欣, 吉建波, 陈鹏, 薛兴旭, 李耀宗. 基于神经网络的火焰筒开孔直径对燃烧室燃烧性能影响的研究[J]. 汽车安全与节能学报, 2026, 17(1): 88-95.
[4]	邱照玉, 祝小元, 田光宇, 殷国栋. 隐蔽网络攻击下智能网联汽车容错安全控制[J]. 汽车安全与节能学报, 2025, 16(6): 914-922.
[5]	程泽阳, 段奕阳, 杨蒙蒙, 冯忠祥, 王鹤, 朱晓俊, 保丽霞. 基于混合神经网络的交织区危险驾驶与风格的识别[J]. 汽车安全与节能学报, 2025, 16(5): 688-697.
[6]	于谦, 郭圆圆, 杨鸣鹏, 张玉婷. 基于跟驰对的CO₂排放特性的生态车辆跟驰策略[J]. 汽车安全与节能学报, 2025, 16(4): 577-586.
[7]	王琳, 陈清华, 业红玲, 王鹏飞, 徐驰, 钱爱文. 基于神经网络自适应MPC智能车辆轨迹跟踪仿真[J]. 汽车安全与节能学报, 2025, 16(4): 638-647.
[8]	房熙博, 宁一高, 赵轩, 周猛. 基于SQP和GRNN的商用客车动力学参数自适应辨识[J]. 汽车安全与节能学报, 2025, 16(4): 648-656.
[9]	虞安军, 励英迪, 杨哲懿, 付崇宇, 童蔚苹, 余佳, 刘云海, 刘志远. 基于多维注意力机制的高速公路交通流量预测方法[J]. 汽车安全与节能学报, 2025, 16(3): 463-469.
[10]	荣海, 蒋建中, 姚再起, 马凯, 杜柯南. 基于神经网络代理模型的车身多学科轻量化优化设计[J]. 汽车安全与节能学报, 2025, 16(1): 32-42.
[11]	张立成, 押境田, 彭琨, 杨冉. 车辆急动度对神经网络油耗预测性能影响研究[J]. 汽车安全与节能学报, 2025, 16(1): 117-126.
[12]	高凯, 刘健, 刘林鸿, 刘欣宇, 张金来, 杜荣华. 基于LSTM-多头混合注意力的可解释换道意图预测[J]. 汽车安全与节能学报, 2024, 15(5): 763-773.
[13]	张晨, 刘畅, 赵津, 王广玮, 许庆. 基于多尺度注意力机制的实时激光雷达点云语义的分割[J]. 汽车安全与节能学报, 2024, 15(4): 591-601.
[14]	王居闯, 曹清林, 邱睿, 宋刘伟, 郭平安, 赵港. 在侧面碰撞中电动汽车电池模块破损的预测[J]. 汽车安全与节能学报, 2024, 15(2): 169-177.
[15]	魏孟, 王桥, 叶敏, 廉高棨, 徐信芯. 基于dropout-MC递归神经网络的锂电池剩余寿命预测[J]. 汽车安全与节能学报, 2022, 13(3): 541-549.