Fatigue driving state detection method based on improved EfficientDet

doi:10.3969/j.issn.1674-8484.2022.04.006

Abstract

Abstract:

A visual detection method was proposed for drivers’ fatigue driving state based on the improved EffcientDet deep learning network to improve the accuracy and efficiency of visual detection and to reduce the requirements of hardware configuration. An EfficientNet network was constructed for extracting driver facial image features by using the deep separable convolution and the visual attention mechanism. An EffcientDet driver state detection network was constructed by using the bidirectional feature pyramid network and the k-means prior frame clustering. The PERCLOS drowsy degree measurement coefficient was used to judge driver fatigue state. Three EfficientDet networks, YOLO V3, and Faster-RCNN were compared and analyzed with different depths, widths and input resolutions. The results show that the improved efficientdet-D2 algorithm has the best efficiency among these algorithms with an average accuracy of 97.92%, a recall rate of 96.75%, an error rate less than 2.39%, and a missed rate less than 1.78%.

Key words: deep learning, image recognition, fatigue driving, state detection, improved EfficientDet deep learning networks

CLC Number:

U461.99

SONG Wei, ZHANG Guangde. Fatigue driving state detection method based on improved EfficientDet[J]. Journal of Automotive Safety and Energy, 2022, 13(4): 651-658.

Figures/Tables 12

References 21

[1]	公安部. 2020年全国新注册登记机动车3328万辆, 新能源汽车保有量492万辆[J]. 商用汽车, 2021(1): 9-10.
	Ministry of Public Security. In 2020, 33.28 million new motor vehicles will be registered nationwide, and the number of new energy vehicles will be 4.92 million[J]. *Commercial Vehi*, 2021(1): 9-10. (in Chinese)
[2]	公安部. 2020年上半年全国机动车保有量达3.6亿辆[J]. 摩托车技术, 2020(8): 25.
	Ministry of Public Security. In the first half of 2020, there were 360 million motor vehicles in China[J]. *Motorcycle Tech*, 2020(8): 25. (in Chinese)
[3]	吴联栩. 驾驶员疲劳驾驶与侥幸心理对策调整[J]. 科学咨询, 2021(1):38.
	WU Lianyu. Driver fatigue driving and fluke psychological countermeasures adjustment[J]. *Scientific Consulting*, 2021(1): 38. (in Chinese)
[4]	王淑宁, 张雄伟, 张宇帆. 运输车辆安全驾驶行为的分析[J]. 内江科技, 2020, 41(1): 88-89.
	WANG Shuning, ZHANG Xiongwei, ZHANG Yufan. Analysis of safe driving behavior of transport vehicles[J]. *Neijiang Technology*, 2020, 41(1): 88-89. (in Chinese)
[5]	张俊星. 浅谈汽车驾驶员的人为因素与交通安全[J]. 时代汽车, 2019(16): 43-44.
	ZHANG Junxing. On the human factors and traffic safety of automobile drivers[J]. *Auto Time*, 2019(16): 43-44. (in Chinese)
[6]	王政, 汪军. 不同姿态下基于多特征融合的疲劳状态检测方法[J]. 重庆工商大学学报(自然科学版), 2021, 38(6): 26-33.
	WANG Zheng, WANG Jun. Fatigue state detection method based on multi-Feature fusion under different postures[J]. J Chongqing Tech Busi Uni (Nat Sci Ed), 2021, 38(6): 26-33. (in Chinese)
[7]	李昭慧, 张玮良. 基于改进YOLOv4算法的疲劳驾驶检测[J]. 电子测量技术, 2021, 44(13): 73-78.
	LI Zhaohui, ZHANG Weiliang. Fatigue driving detection based on improved YOLOv4 algorithm[J]. *Electronic Meas Tech*, 2021, 44(13): 73-78. (in Chinese)
[8]	王俊杰, 汪洋堃, 张峰, 等. 基于CNN和SVM的疲劳驾驶闭眼特征实时检测[J]. 计算机系统应用, 2021, 30(6): 118-126.
	WANG Junjie, WANG Yangkun, ZHANG Feng, et al. Real-time detection of eye closing features in tired driving based on CNN and SVM[J]. *Computer Syst Appl*, 2021, 30(6): 118-126. (in Chinese)
[9]	马雪婷, 费树岷. 基于面部特征与深度学习的疲劳驾驶状态检测研究[J]. 电子测试, 2021(11): 33-36.
	MA Xueting, FEI Shumin. Fatigue driving state detection based on facial features and deep learning[J]. *Electronic Testing*, 2021(11): 33-36. (in Chinese)
[10]	纪世雨. 一种改进型KCF的疲劳驾驶检测方法[J]. 计算机测量与控制, 2022(1): 52-59.
	JI Shiyu. An improved KCF fatigue driving detection method[J]. *Computer Meas Contrl*, 2022(1): 52-59. (in Chinese)
[11]	施冬梅, 肖锋. 基于改进长短时记忆网络的驾驶行为检测方法研究[J]. 汽车工程, 2021, 43(8): 1203-1209+1262.
	SHI Dongmei, XIAO Feng. Driving behavior detection method based on improved short and long duration memory network[J]. *Auto Engineering*, 2021, 43(8): 1203-1209+1262. (in Chinese)
[12]	程泽, 林富生, 靳朝, 等. 基于轻量化卷积神经网络的疲劳驾驶检测[J]. 重庆理工大学学报(自然科学), 2022, 36(2): 142-150.
	CHENG Ze, LIN Fusheng, QIN Chao, et al. Fatigue driving Detection based on lightweight convolutional neural network[J]. J Chongqing Univ Tech (Nat Sci), 2022, 36(2): 142-150. (in Chinese)
[13]	李小平, 白超. 基于深度学习的司机疲劳驾驶检测方法研究[J]. 铁道学报, 2021, 43(6): 78-87.
	LI Xiaoping, BAI Chao. Research on driver fatigue detection method based on deep learning[J]. *J Railway*, 2021, 43(6): 78-87. (in Chinese)
[14]	何艳清. 基于深度学习的疲劳检测方法研究[D]. 北京: 北京建筑大学, 2021.
	HE Yanqing. Research on fatigue detection method based on deep learning[D]. Beijing: Beijing University of Civil Engineering and Architecture, 2021. (in Chinese)
[15]	陈强, 戴军. 农机驾驶人员疲劳驾驶行为检测研究[J]. 中国农机化学报, 2021, 42(5): 148-152. doi: 10.13733/j.jcam.issn.2095-5553.2021.05.20
	CHEN Qiang, DAI Jun. Study on fatigue driving behavior detection of agricultural machinery drivers[J]. *J Chin Agri Mech*, 2021, 42(5): 148-152. (in Chinese)
[16]	TAN Mingxing, LE Q V. EfficientNet: Rethinking model scaling for convolutional neural networks[C]// Proc 36th Int’l Conf Machine Learning (PMLR), 2019, 97: 6105-6114.
[17]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]// Proc IEEE Conf Computer Vision Pattern Recog (CVPR), 2016: 770-778.
[18]	HU Jie, LI Shen, Samuel A, et al. Squeeze-and-excitation networks[C]// Proc IEEE Conf Computer Vision Pattern Recog (CVPR), 2018: 7132-7141.
[19]	TAN Mingxing, PANG Ruoming, LE Q V. EfficientDet: Scalable and efficient object detection[C]// Proc IEEE Conf Computer Vision Pattern Recog (CVPR), 2020: 10781-10790.
[20]	LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[C]// Proc IEEE Conf Computer Vision Pattern Recog (CVPR), 2018: 8759-8768.
[21]	Dinges D F, Grace R. PERCLOS: A valid psychophysiological measure of alertness as assessed by psychomotor vigilance[J]. *Tech Brief*, 1998, 31(5): 1237-1252.

数据集	样本数				总计
数据集	正常	闭眼	张嘴	低头	总计
训练	4 421	4 394	4 399	4 411	17 625
测试	397	399	389	392	1 577
验证	380	380	380	380	1 520

数据集	样本数				总计
数据集	正常	闭眼	张嘴	低头	总计
训练	4 421	4 394	4 399	4 411	17 625
测试	397	399	389	392	1 577
验证	380	380	380	380	1 520

	配置内容	实验条件
基础设置	学习率	1×10^-4
	学习衰减速率	5×10^-5
	输入图像大小	512×512
	动量	0.9
	损失函数	SGD
训练设置	批次大小	8
训练设置	训练回合	400
EarlyStopping	检测器	Valid loss
	最小变化率	0
	耐心值	10
训练环境	图形处理器	Nvidia RTX 2080Ti
	平台	Python 3.7
	工具箱	Pytorch

	配置内容	实验条件
基础设置	学习率	1×10^-4
	学习衰减速率	5×10^-5
	输入图像大小	512×512
	动量	0.9
	损失函数	SGD
训练设置	批次大小	8
训练设置	训练回合	400
EarlyStopping	检测器	Valid loss
	最小变化率	0
	耐心值	10
训练环境	图形处理器	Nvidia RTX 2080Ti
	平台	Python 3.7
	工具箱	Pytorch

疲劳程度	Perclos	闭眼频率/ s^-1
正常驾驶	< 0.25	0
轻微疲劳驾驶	> 0.25	5
严重疲劳驾驶	> 0.25	10