低光照环境下基于面部特征点的疲劳驾驶检测技术

doi:10.3969/j.issn.1674-8484.2022.02.008

摘要/Abstract

摘要：

提出了一种新型在低光照环境下有较高适应性和识别精度的疲劳驾驶检测技术。用深度视觉传感器获取驾驶员驾驶图像,用人脸跟踪算法实时提取面部特征点数据,基于最小二乘法对眼睛和嘴巴轮廓进行曲线拟合。计算眼睛和嘴巴开合度归一化指标,提取了眨眼频率、眨眼平均时长、眼睛闭合总时长、打哈欠频率、打哈欠总时长、低抬头频率等6个疲劳识别特征数据。基于数据统计序列的卷积神经网络算法,建立识别模型,构建疲劳状态检测系统。实验表明：本文算法在低光照环境下的疲劳驾驶识别精度达到了90%,识别时间约为130 ms。

关键词: 汽车安全, 疲劳驾驶识别, 深度视觉传感器, 卷积神经网络, 低光照环境, 面部特征点, 归一化指标

Abstract:

A new fatigue driving detection technology was proposed with high adaptability and recognition accuracy in low light environment. A depth vision sensor was used to obtain the driver’s driving image in real time;with extracted the face feature point data in real time through a face tracking algorithm to fit a eyes and mouth contour with a least square method.The normalized indexes were calculated for the eye and mouth opening and closing to extract six fatigue recognition feature data including the blink frequency, the average blink time, the total eye closing time, the yawning frequency, the total yawning time, and the low head up frequency. A recognition model was established based on the convolution neural network algorithm of data statistical sequence to construct a fatigue state detection system. Experiments show that in low light environment, this algorithm has a accuracy of 90% for fatigue driving recognition with a recognition time of about 130 ms.

Key words: automotive safety, fatigue driving identification, depth vision sensor, convolutional neural network, low light environment, face feature point, normalized index

中图分类号:

TP391

朱艳, 谢忠志, 于雯, 李曙生, 张逊. 低光照环境下基于面部特征点的疲劳驾驶检测技术[J]. 汽车安全与节能学报, 2022, 13(2): 282-289.

ZHU Yan, XIE Zhongzhi, YU Wen, LI Shusheng, ZHANG Xun. Fatigue-driving detect-technology in low light environment based on facial feature points[J]. Journal of Automotive Safety and Energy, 2022, 13(2): 282-289.

图/表 16

图1 Face Tracking人脸跟踪流程图

图2 低可见度下RGB图像和深度图像人脸跟踪效果

图3 疲劳识别所用面部特征点分布

图4 Kinect V2头部转角识别示意图

图5 面部特征点数据坐标系统

图6 面部特征坐标变换过程

图7 眼睛轮廓拟合曲线和外接圆

图8 嘴巴轮廓拟合曲线和外接圆

图9 眼睛开合度归一化指标ηNAE曲线图

图10 嘴巴开合度归一化指标ηNAM曲线图

疲劳等级	f_b min^-1	t_b s	t_c s	f_y min^-1	t_y s	f_h min^-1
不	> 15	< 0.3	< 2	< 1	< 0.5	< 3
轻度	10~15	0.3~0.5	2~3	1~2	0.5~2	3~6
中度	6~10	0.5~1	3~5	2~4	2~5	6~10
严重	< 6	> 1	> 5	> 4	> 5	> 10

表1 疲劳等级划分和识别特征数据之间的关系

图11 卷积神经网络疲劳识别模型结构

图12 卷积神经网络疲劳识别模型训练过程

图13 疲劳驾驶实验所处的光照环境图

年龄/ 岁	人数		样本数/ 个				模拟时长/ min
年龄/ 岁	男	女	正常驾驶	轻度疲劳	中度疲劳	严重疲劳	正常驾驶	轻度疲劳	中度疲劳	严重疲劳
20~30	6	4	10	10	10	10	30	30	30	30
30~40	4	2	14	10	8	8	42	30	24	24
40~45	2	2	16	10	7	7	48	30	21	21

表2 实验对象年龄、性别、不同疲劳状态下的模拟时长

算法模型	正常光照情况		低光照情况
算法模型	准确率/ %	识别时间/ ms	准确率/ %	识别时间/ ms
TVA	66	69	62	78
SVM	80	282	74	272
Ada-Boost	87	242	85	253
DT	75	206	77	224
KNN	78	185	70	181
CNN	91	125	90	132

表3 正常光照和低光照环境下各算法模型识别结果

参考文献 25

[1]	徐军莉, 闵建亮, 胡剑锋, 等. 定性推理生成器在驾驶疲劳检测中的应用[J]. 汽车安全与节能学报, 2018, 9(1): 32-40.
	XU Junli, MIN Jianliang, HU Jianfeng, et al. Application of qualitative reasoning generator in driving fatigue detection[J]. J Auto Safe Energy, 2018, 9(1): 32-40. (in Chinese)
[2]	Dias N S, Carmo J P, Mendes P M, et al. Wireless instrumentation system based on dry electrodes for acquiring EEG signals[J]. Medi Eng Phys, 2012, 34(7): 972-981.
[3]	李江天, 李敏, 宋战兵. 基于多源生理信号的驾驶疲劳检测[J]. 物流技术, 2018, 37(2): 78-83.
	LI Jiangtian, LI Min, SONG Zhanbing. Driving fatigue detection based on multi-source physiological signals[J]. Logistics Technology, 2018, 37 (2): 78-83. (in Chinese)
[4]	Borghini G, Astolfi L, Vecchiato G, et al. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness[J]. Neurosci Biobeha Rev, 2012, 44: 58-75. doi: 10.1016/j.neubiorev.2012.10.003 URL
[5]	Friedrichs F, YANG Bin. Drowsiness monitoring by steering and lane data based features under real driving conditions[C]// Signal Proc Conf, 2010, Euro IEEE, 2010: 209-213.
[6]	刘军, 王利明, 聂斐, 等. 基于转向盘转角的疲劳驾驶检测方法研究[J]. 汽车技术, 2016(4): 42-45.
	LIU Jun, WANG Liming, NIE Fei, et al. Research on fatigue driving detection method based on steering wheel angle[J]. Automotive Tech, 2016(4): 42-45. (in Chinese)
[7]	李作进, 李仁杰, 李升波. 基于方向盘转角近似熵与复杂度的驾驶人疲劳状态识别[J]. 汽车安全与节能学报, 2016, 7(3): 279-284.
	LI Zuojin, LI Renjie, LI Shengbo. Driver fatigue state recognition based on approximate entropy and complexity of steering wheel angle[J]J Auto Safe Energy, 2016, 7(3): 279-284. (in Chinese)
[8]	LI Zuojin, CHEN Liukui, PENG Jun, et al. Automatic detection of driver fatigue using driving operation information for transportation safety[J]. Sensors, 2017, 17(6): 1-12. doi: 10.3390/s17010001 URL
[9]	Sikander G, Anwar S. Driver fatigue detection systems: A review[J]. IEEE Trans Intell Transp Syst, 2019, 20(6): 2339-2352. doi: 10.1109/TITS.2018.2868499 URL
[10]	樊星, 刘占文, 林杉, 等. 融合人眼特征与深度学习的疲劳驾驶检测模型[J]. 计算机工程, 2021, 47(8): 243-250.
	FAN Xing, LIU Zhanwen, LIN Shan, et al. Fatigue driving detection model integrating human eye features and deep learning[J]. Compu Engirg, 2021, 47(8): 243-250. (in Chinese)
[11]	Hilal A, Ali A, Waleed A. Modular design of fatigue detection in naturalistic driving environments[J]. Accid Anal Preve, 2018, 120: 188-194.
[12]	ZHANG Fang, SU Jingjing. Driver fatigue detection based on eye state recognition[C]// 2017: 105-110.
[13]	KIM Whui, JUNG Woo-Sung, CHOI Hyun-Kyun. Lightweight driver monitoring system based on multi-task mobilenets[J]. Sensors, 2019, 19(14): 1-18. doi: 10.3390/s19010001 URL
[14]	胡习之, 黄冰瑜. 基于面部特征分析的疲劳驾驶检测方法[J]. 科学技术与工程, 2021, 21(4): 1629-1636.
	HU Xizhi, HUANG Bingyu. Fatigue driving detection method based on facial feature analysis[J]. Sci Tech Engirg, 2021, 21(4): 1629-1636. (in Chinese)
[15]	方斌, 徐硕, 冯晓锋. 面向ARM平台的自标定驾驶员疲劳检测方法[J]. 汽车安全与节能学报, 2020, 11(1): 71-78.
	FANG Bin, XU Shuo, FENG Xiaofeng. Self calibration driver fatigue detection method for arm platform[J]. J Auto Safe Energy, 2020, 11(1): 71-78. (in Chinese)
[16]	娄平, 杨欣, 胡辑伟, 等. 基于边缘计算的疲劳驾驶检测方法[J]. 计算机工程, 2021, 47(7): 13-20+29.
	LOU Ping, YANG Xin, HU Jiwei, et al. Fatigue driving detection method based on edge calculation[J]. Comp Engineering, 2021, 47(7): 13-20+29. (in Chinese)
[17]	ZHANG Kaipeng, ZHANG Zhanpeng, LI Zhifeng. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Proces Letters, 2016, 23(10): 1-5. doi: 10.1109/LSP.2016.2516438 URL
[18]	XIANG Jia, ZHU Gengming. Joint face detection and facial expression recognition with MTCNN[C]// 2017 4th Int’l Conf Info Sci Control Engi, Changsha, China, 2017: 424-427.
[19]	LIU Weiwei, SUN Haixin, Shen Weijie. Driver fatigue detection through pupil detection and yawing analysis[C]// 2010 Int’l Conf’Bioinfo Biomed Tech, New York, 2010: 404-407.
[20]	HU Siquan, LIN Zhenye. Fatigue driving detection based on machine learning and image processing technology[C]// 3rd Ann Int’l Conf Info Syst Arti Intel (ISAI), Beijing, China, 2018: 1-6.
[21]	Reddy B, KIMYehoon, YUNSojung. Real-time driver drowsiness detection for embedded system using model compression of deep neural networks[C]// The 2017 IEEE Conf Computer Vision Pattern Recog (CVPR), Honolulu, USA, 2017: 438-445.
[22]	朱艳, 李曙生, 谢忠志. 基于FCM聚类和卷积神经网络的跌倒识别算法[J]. 数据采集与处理, 2021, 36(04): 746-755.
	ZHU Yan, LI Shusheng, XIE Zhongzhi. Fall recognition algorithm based on FCM clustering and convolutional neural network[J]. Data Acquisition & Processing, 2021, 36 (4): 746-755. (in Chinese)
[23]	Ngxande M, Tapamo J R. Driver drowsiness detection using behavioral measures and machine learning techniques: A review of state-of-art techniques[C]// Pattern Recog Asso South Africa Robo Mechatronics (PRASA- RobMech), Bloemfontein, South Africa, 2017: 156-161.
[24]	Akalya C, Mandi S R, Reddy E, et al. Fatigue detection using raspberry Pi 3[J]. Int’l J Engin Tech, 2018, 7(2): 29-32.
[25]	Celona L, Mammana L. A multi-task CNN framework for driver face monitoring[C]// 2018 IEEE 8th Int’l Conf Consumer Electr, Berlin, 2018: 612-616.