基于多源传感器与导航地图的端到端自动驾驶方法

doi:10.3969/j.issn.1674-8484.2022.04.015

汽车安全与节能学报 ›› 2022, Vol. 13 ›› Issue (4): 738-749.DOI: 10.3969/j.issn.1674-8484.2022.04.015

基于多源传感器与导航地图的端到端自动驾驶方法

朱波¹^,²(), 张纪伟¹(), 谈东奎¹^,²^,^*(), 胡旭东¹

1.合肥工业大学汽车工程技术研究院，合肥市 230009，中国
2.合肥工业大学智能制造技术研究院，合肥市 230051，中国

收稿日期:2022-05-21 修回日期:2022-07-25 出版日期:2022-12-31 发布日期:2023-01-01
通讯作者: 谈东奎
作者简介:*谈东奎，助理研究员。E-mail：tandongkui@126.com。
朱波(1980—)，男(汉)，湖北，副研究员。E-mail: zhubo@hfut.edu.cn。
张纪伟(1993—)，男(汉)，河南，硕士研究生。E-mail: z18336345753@163.com。
基金资助:
安徽省科技重大专项(201903a05020016);安徽省重点研究与开发计划(202004b11020002);安徽省自然科学基金(2208085QE153)

End-to-end autonomous driving method based on multi-source sensor and navigation map

ZHU Bo¹^,²(), ZHANG Jiwei¹(), TAN Dongkui¹^,²^,^*(), HU Xudong¹

1. Automotive Research Institute, Hefei University of Technology, Hefei 230009, China
2. Intelligent Manufacturing Institute, Hefei University of Technology, Hefei 230009, China

Received:2022-05-21 Revised:2022-07-25 Online:2022-12-31 Published:2023-01-01
Contact: TAN Dongkui

摘要/Abstract

摘要：

提出了一种基于多源传感器与导航地图的多端输入-单端输出(端到端)自动驾驶决策控制模型，以弥补现有端到端自动驾驶方法中基于深度神经网络(DNN)的PilotNet模型在主动避障行驶和交叉路口通行方面的不足。该模型的传感器数据输入端包括：单目前视摄像头、360( ° )多线激光雷达(LiDAR)所得二维俯视图、精准定位的局部导航地图等3部分；车辆控制命令输出端为方向盘转向角。进行了多工况仿真和实车试验。结果表明：与PilotNet模型相比，该模型的方向盘转向角均方根误差(RMSE)值下降了37%；因而，该模型具备主动避障和交叉路口通行的能力。

关键词: 自动驾驶车辆, 端到端自动驾驶, 神经网络, 摄像头, 导航地图, 激光雷达(LiDAR)

Abstract:

This paper proposed a multiple-input single-output (end-to-end) autonomous driving decision control model based on multi-source sensors and navigation map to make up for the shortcomings of existing end-to-end autonomous driving methods in active obstacle avoidance and passing through intersection by using the PilotNet model based on Deep Neural Network (DNN). The sensor data in model’s input end consisted of three parts: a monocular front view camera, a 2-D top view obtained by 360° multi-layer Light-Laser Detection and Ranging (LiDAR), and a local navigation map based on accurate positioning. The model’s output end generated the steering wheel angle, which was a vehicle control command. Some multi-condition simulations and real vehicle tests were conducted. The results show that the root mean square error (RMSE) of steering wheel angle decreases by 37% compared with that by using the PilotNet model. Therefore, the model has the ability of active obstacle avoidance and intersection passing.

Key words: autonomous vehicles, end-to-end autonomous driving, neural networks, camera, navigation maps, light-laser detection and ranging (LiDAR)

中图分类号:

U467.1+3

朱波, 张纪伟, 谈东奎, 胡旭东. 基于多源传感器与导航地图的端到端自动驾驶方法[J]. 汽车安全与节能学报, 2022, 13(4): 738-749.

ZHU Bo, ZHANG Jiwei, TAN Dongkui, HU Xudong. End-to-end autonomous driving method based on multi-source sensor and navigation map[J]. Journal of Automotive Safety and Energy, 2022, 13(4): 738-749.

图/表 26

图1 3种端到端决策模型架构

整车装备质量，m	1.532 t
前轴到车辆质心的距离, l_f	1.21 m
后轴到车辆质心的距离, l_r	1.5 m
质心高度, h_cog	0.5 m
车辆的宽度, w_v	2.07 m
车辆的长度 l	4.63 m

表1 车辆主要参数

图2 LiDAR坐标系和图像坐标系

图3 16线激光雷达俯视图

图4 局部导航地图截取流程

图5 局部导航地图旋转前后的对比

图6 车辆初始化偏离车道中心

图7 Pure Pursuit算法跟踪模型

图8 Dataset-C转向角统计

图9 Model-C模型训练过程示意图

图10 Model-C模型训练Loss变化曲线

网络模型	RMSE / (°)
PilotNet	5.67
Model-A	2.81
Model-B	3.23
Model-C	3.57

表2 不同模型的性能对比

图11 车道保持验证场景

图12 基础模型/Model-A/PilotNet转向角预测

图13 Model-A/B/C转向角预测

图14 避障验证场景图

图15 避障场景下Model-A/B/C的车辆轨迹

图16 车辆障碍物场景下基于Dataset-B的Model-A/B的车辆轨迹

图17 交叉路口验证场景

图18 全局导航路径nav-1和nav-2

图19 Model-B/C预测的车辆行驶轨迹对比

模型	输入数据	t / ms	车道保持功能	主动避障行驶	通过交叉路口
PilotNet	摄像头	3.9	√	×	×
Model-A	摄像头	5.6	√	×	×
Model-B	摄像头 + LiDAR	9.8	√	√	×
Model-C	摄像头 + LiDAR + 导航地图	12.7	√	√	√

表3 不同模型间的功能表现对比

图20 智能驾驶电动汽车试验平台

图21 园区道路俯视图及测试路线

图22 真实场景道路图像示例

图23 人类驾驶员与Model-C的转向角对比

参考文献 21

[1]	陈颖. 自动驾驶技术研究现状及发展趋势[J]. 能源技术与管理, 2021, 46(3): 34-37.
	CHEN Ying. Research status and development trend of automatic driving technology[J]. *Energy Tech Manag*, 2021, 46(3): 34-37. (in Chinese)
[2]	Broggi A, Cerri P, Debattisti S, et al. PROUD Public road urban driverless-car test[J]. *IEEE Trans Intell Transp Syst*, 2015, 16(6): 3508-3519.
[3]	Bojarski M, Yeres P, Choromanska A, et al. Explaining how a deep neural network trained with end-to-end learning steers a car[EB/OL]. (2017-04-25), https://arxiv.org/abs/1704.07911.
[4]	WANG Tinghan, LUO Yugong, LIU Jinxin, et al. End-to-end self-driving approach independent of irrelevant roadside objects with auto-encoder[J]. *IEEE Trans Intell Transp Syst*, 2020, 23(1): 641-650.
[5]	Pomerleau D A. Alvinn: An autonomous land vehicle in a neural network[C]// Adva Neural Info Proc Syst. San Francisco: Morgan Kaufmann Publishers Inc, 1989: 305-313.
[6]	LeCun Y, Muller U, Ben J, et al. Off-road obstacle avoidance through end-to-end learning[C]// Int’l Conf Neural Info Proc Syst, Vancouver, 2005: 1-8.
[7]	Bojarski M, Testa D, Dworakowski D, et al. End to end learning for self-driving cars[EB/OL]. (2016-04-25), http://arxiv.org/abs/1604.07316.
[8]	XU Huazhe, GAO Yang, YU Fisher, et al. End-to-end learning of driving models from large-scale video datasets[C]// Proc IEEE Conf Computer Vision Pattern Recog, IEEE, Honolulu, 2017: 2174-2182.
[9]	胡学敏, 童秀迟, 郭琳, 等. 基于深度视觉注意神经网络的端到端自动驾驶模型[J]. 计算机应用, 2020, 40(7): 1926-1931. doi: 10.11772/j.issn.1001-9081.2019112054
	HU Xuemin, TONG Xiuchi, GUO Lin, et al. End-to-end autonomous driving model based on deep visual attention neural network[J]. *Computer Appl*, 2020, 40(7): 1926-1931. (in Chinese)
[10]	Codevilla F, Mller M, Lpez A, et al. End-to-end driving via conditional imitation learning[C]// 2018 IEEE Int’l Conf Robo Automation (ICRA). Brisbane, 2018: 4693-4700.
[11]	Sallab A, Abdou M, Perot E, et al. Deep reinforcement learning framework for autonomous driving[J]. *Electronic Imaging*, 2017(19): 70-76.
[12]	CHEN Siyuan, WANG Meiling, SONG Wenjie, et al. Stabilization approaches for reinforcement learning-based end-to-end autonomous driving[J]. *IEEE Trans Vehi Tech*, 2020, 69(5): 4740-4750.
[13]	SONG Wenjie, LIU Shixian, LI Yujun, et al. Smooth actor-critic algorithm for end-to-end autonomous driving[C]// 2020 Ame Control Conf, ACC, Colorado: 2020: 3242-3248.
[14]	Abdou M, Kamal H, El-Tantawy S, et al. End-to-end deep conditional imitation learning for autonomous driving[C]// 2019 31st Int’l Conf Micro- Electronics (ICM), IEEE, Beijing, 2020: 346-350.
[15]	卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述[J]. 数据采集与处理, 2016, 31(1): 1-17.
	LU Hongtao, ZHANG Qinchuan. A review of the application of deep convolutional neural networks in computer vision[J]. *Data Acqu Proce*, 2016, 31(1): 1-17. (in Chinese)
[16]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10), https://arxiv.org/abs/1409.1556.
[17]	Toth L. Phone recognition with deep sparse rectifier neural networks[C]// IEEE Acoustics Speech Signal Proc (ICASSP), IEEE, California: 2013: 6985-6989.
[18]	CHEN Zhilu, HUANG Xinming. End-to-end learning for lane keeping of self-driving cars[C]// 2017 IEEE Intell Vehi Symp (IV). IEEE, Los Angeles, 2017: 1856-1860.
[19]	Jhung J, Bae I, Moon J, et al. End-to-end steering controller with CNN-based closed-loop feedback for autonomous vehicles[C]// 2018 IEEE Intell Vehi Symp (IV). IEEE, Changshu, 2018: 617-622.
[20]	Mihalea A, Samoilescu R, Nica A, et al. End-to-end models for self-driving cars on UPB campus roads[C]// 2019 IEEE 15th Int’l Conf Intell Computer Commu Proc (ICCP). IEEE, Tokyo, 2019: 35-40.
[21]	YANG Zhengyuan, ZHANG Yixuan, YU Jerry, et al. End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perceptions[C]// 2018 24th Int’l Conf Pattern Recog (ICPR). IEEE, Beijing, 2018: 2289-2294.

基于多源传感器与导航地图的端到端自动驾驶方法

End-to-end autonomous driving method based on multi-source sensor and navigation map

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 26

参考文献 21

相关文章 15

编辑推荐

Metrics

本文评价

期刊信息

在线期刊

作者中心

审稿中心

联系我们

[1]	魏孟, 王桥, 叶敏, 廉高棨, 徐信芯. 基于dropout-MC递归神经网络的锂电池剩余寿命预测[J]. 汽车安全与节能学报, 2022, 13(3): 541-549.
[2]	朱艳, 谢忠志, 于雯, 李曙生, 张逊. 低光照环境下基于面部特征点的疲劳驾驶检测技术[J]. 汽车安全与节能学报, 2022, 13(2): 282-289.
[3]	胡远志, 蒋涛, 刘西, 施友宁. 基于双流自适应图卷积神经网络的行人过街意图识别[J]. 汽车安全与节能学报, 2022, 13(2): 325-332.
[4]	王宝琳, 夏怀成, 董倩倩. 车辆转弯工况下间接式胎压监测系统脉冲数的修正[J]. 汽车安全与节能学报, 2021, 12(4): 507-515.
[5]	张平, 迟志诚, 陈一凡, 惠飞. 用于自动驾驶车辆的融合注意力机制多目标跟踪算法[J]. 汽车安全与节能学报, 2021, 12(4): 516-521.
[6]	李耀华, 范吉康, 刘洋, 何杰, 李泽田, 潘绍飞. 自适应双时域参数MPC的智能车辆路径规划与跟踪控制[J]. 汽车安全与节能学报, 2021, 12(4): 528-539.
[7]	冯敏健, 张辉, 巨志扬, 许庆. 基于LGSVL/ Apollo的网络延迟攻击下自动驾驶车辆定位估算[J]. 汽车安全与节能学报, 2021, 12(1): 62-69.
[8]	罗浩文, 吴小娟, 张铭涛. 固体氧化物燃料电池系统的复合优化在线控制策略[J]. 汽车安全与节能学报, 2020, 11(4): 553-559.
[9]	谢辉, 聂振华, 陈韬. 基于 Bayes 正则化的柴油机神经网络燃烧预测模型[J]. 汽车安全与节能学报, 2020, 11(3): 345-354.
[10]	申恩恩，胡玉梅，陈光，罗攀，朱浩. 智能驾驶实时目标检测的深度卷积神经网络[J]. JASE, 2020, 11(1): 111-116.
[11]	张晨阳，李曙光，李耀华，姚进,祝磊，王敏. 基于驾驶模拟器和 fNIRS 的与晕动病相关脑活动区的差异性[J]. JASE, 2019, 10(4): 433-442.
[12]	胡远志，刘俊生，何佳，肖航，宋佳. 基于激光雷达点云与图像融合的车辆目标检测方法[J]. JASE, 2019, 10(4): 451-458.
[13]	戴荣健，丁川，鹿应荣，赵福全. 自动驾驶环境下车辆轨迹及交通信号协同控制[J]. JASE, 2019, 10(4): 531-539.
[14]	李升波, 关阳，侯廉，高洪波，段京良，梁爽，汪玉，成波，李克强，任伟，李骏. 深度神经网络的关键技术及其在自动驾驶领域的应用[J]. JASE, 2019, 10(2): 119-145.
[15]	白傑，郝培涵，陈思汉. 用轻量化卷积神经网络图像语义分割的交通场景理解[J]. JASE, 2018, 9(4): 433-440.