Fatigue-driving detect-technology in low light environment based on facial feature points

doi:10.3969/j.issn.1674-8484.2022.02.008

Abstract

Abstract:

A new fatigue driving detection technology was proposed with high adaptability and recognition accuracy in low light environment. A depth vision sensor was used to obtain the driver’s driving image in real time;with extracted the face feature point data in real time through a face tracking algorithm to fit a eyes and mouth contour with a least square method.The normalized indexes were calculated for the eye and mouth opening and closing to extract six fatigue recognition feature data including the blink frequency, the average blink time, the total eye closing time, the yawning frequency, the total yawning time, and the low head up frequency. A recognition model was established based on the convolution neural network algorithm of data statistical sequence to construct a fatigue state detection system. Experiments show that in low light environment, this algorithm has a accuracy of 90% for fatigue driving recognition with a recognition time of about 130 ms.

Key words: automotive safety, fatigue driving identification, depth vision sensor, convolutional neural network, low light environment, face feature point, normalized index

CLC Number:

TP391

ZHU Yan, XIE Zhongzhi, YU Wen, LI Shusheng, ZHANG Xun. Fatigue-driving detect-technology in low light environment based on facial feature points[J]. Journal of Automotive Safety and Energy, 2022, 13(2): 282-289.

Figures/Tables 16

References 25

[1]	徐军莉, 闵建亮, 胡剑锋, 等. 定性推理生成器在驾驶疲劳检测中的应用[J]. 汽车安全与节能学报, 2018, 9(1): 32-40.
	XU Junli, MIN Jianliang, HU Jianfeng, et al. Application of qualitative reasoning generator in driving fatigue detection[J]. J Auto Safe Energy, 2018, 9(1): 32-40. (in Chinese)
[2]	Dias N S, Carmo J P, Mendes P M, et al. Wireless instrumentation system based on dry electrodes for acquiring EEG signals[J]. Medi Eng Phys, 2012, 34(7): 972-981.
[3]	李江天, 李敏, 宋战兵. 基于多源生理信号的驾驶疲劳检测[J]. 物流技术, 2018, 37(2): 78-83.
	LI Jiangtian, LI Min, SONG Zhanbing. Driving fatigue detection based on multi-source physiological signals[J]. Logistics Technology, 2018, 37 (2): 78-83. (in Chinese)
[4]	Borghini G, Astolfi L, Vecchiato G, et al. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness[J]. Neurosci Biobeha Rev, 2012, 44: 58-75. doi: 10.1016/j.neubiorev.2012.10.003 URL
[5]	Friedrichs F, YANG Bin. Drowsiness monitoring by steering and lane data based features under real driving conditions[C]// Signal Proc Conf, 2010, Euro IEEE, 2010: 209-213.
[6]	刘军, 王利明, 聂斐, 等. 基于转向盘转角的疲劳驾驶检测方法研究[J]. 汽车技术, 2016(4): 42-45.
	LIU Jun, WANG Liming, NIE Fei, et al. Research on fatigue driving detection method based on steering wheel angle[J]. Automotive Tech, 2016(4): 42-45. (in Chinese)
[7]	李作进, 李仁杰, 李升波. 基于方向盘转角近似熵与复杂度的驾驶人疲劳状态识别[J]. 汽车安全与节能学报, 2016, 7(3): 279-284.
	LI Zuojin, LI Renjie, LI Shengbo. Driver fatigue state recognition based on approximate entropy and complexity of steering wheel angle[J]J Auto Safe Energy, 2016, 7(3): 279-284. (in Chinese)
[8]	LI Zuojin, CHEN Liukui, PENG Jun, et al. Automatic detection of driver fatigue using driving operation information for transportation safety[J]. Sensors, 2017, 17(6): 1-12. doi: 10.3390/s17010001 URL
[9]	Sikander G, Anwar S. Driver fatigue detection systems: A review[J]. IEEE Trans Intell Transp Syst, 2019, 20(6): 2339-2352. doi: 10.1109/TITS.2018.2868499 URL
[10]	樊星, 刘占文, 林杉, 等. 融合人眼特征与深度学习的疲劳驾驶检测模型[J]. 计算机工程, 2021, 47(8): 243-250.
	FAN Xing, LIU Zhanwen, LIN Shan, et al. Fatigue driving detection model integrating human eye features and deep learning[J]. Compu Engirg, 2021, 47(8): 243-250. (in Chinese)
[11]	Hilal A, Ali A, Waleed A. Modular design of fatigue detection in naturalistic driving environments[J]. Accid Anal Preve, 2018, 120: 188-194.
[12]	ZHANG Fang, SU Jingjing. Driver fatigue detection based on eye state recognition[C]// 2017: 105-110.
[13]	KIM Whui, JUNG Woo-Sung, CHOI Hyun-Kyun. Lightweight driver monitoring system based on multi-task mobilenets[J]. Sensors, 2019, 19(14): 1-18. doi: 10.3390/s19010001 URL
[14]	胡习之, 黄冰瑜. 基于面部特征分析的疲劳驾驶检测方法[J]. 科学技术与工程, 2021, 21(4): 1629-1636.
	HU Xizhi, HUANG Bingyu. Fatigue driving detection method based on facial feature analysis[J]. Sci Tech Engirg, 2021, 21(4): 1629-1636. (in Chinese)
[15]	方斌, 徐硕, 冯晓锋. 面向ARM平台的自标定驾驶员疲劳检测方法[J]. 汽车安全与节能学报, 2020, 11(1): 71-78.
	FANG Bin, XU Shuo, FENG Xiaofeng. Self calibration driver fatigue detection method for arm platform[J]. J Auto Safe Energy, 2020, 11(1): 71-78. (in Chinese)
[16]	娄平, 杨欣, 胡辑伟, 等. 基于边缘计算的疲劳驾驶检测方法[J]. 计算机工程, 2021, 47(7): 13-20+29.
	LOU Ping, YANG Xin, HU Jiwei, et al. Fatigue driving detection method based on edge calculation[J]. Comp Engineering, 2021, 47(7): 13-20+29. (in Chinese)
[17]	ZHANG Kaipeng, ZHANG Zhanpeng, LI Zhifeng. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Proces Letters, 2016, 23(10): 1-5. doi: 10.1109/LSP.2016.2516438 URL
[18]	XIANG Jia, ZHU Gengming. Joint face detection and facial expression recognition with MTCNN[C]// 2017 4th Int’l Conf Info Sci Control Engi, Changsha, China, 2017: 424-427.
[19]	LIU Weiwei, SUN Haixin, Shen Weijie. Driver fatigue detection through pupil detection and yawing analysis[C]// 2010 Int’l Conf’Bioinfo Biomed Tech, New York, 2010: 404-407.
[20]	HU Siquan, LIN Zhenye. Fatigue driving detection based on machine learning and image processing technology[C]// 3rd Ann Int’l Conf Info Syst Arti Intel (ISAI), Beijing, China, 2018: 1-6.
[21]	Reddy B, KIMYehoon, YUNSojung. Real-time driver drowsiness detection for embedded system using model compression of deep neural networks[C]// The 2017 IEEE Conf Computer Vision Pattern Recog (CVPR), Honolulu, USA, 2017: 438-445.
[22]	朱艳, 李曙生, 谢忠志. 基于FCM聚类和卷积神经网络的跌倒识别算法[J]. 数据采集与处理, 2021, 36(04): 746-755.
	ZHU Yan, LI Shusheng, XIE Zhongzhi. Fall recognition algorithm based on FCM clustering and convolutional neural network[J]. Data Acquisition & Processing, 2021, 36 (4): 746-755. (in Chinese)
[23]	Ngxande M, Tapamo J R. Driver drowsiness detection using behavioral measures and machine learning techniques: A review of state-of-art techniques[C]// Pattern Recog Asso South Africa Robo Mechatronics (PRASA- RobMech), Bloemfontein, South Africa, 2017: 156-161.
[24]	Akalya C, Mandi S R, Reddy E, et al. Fatigue detection using raspberry Pi 3[J]. Int’l J Engin Tech, 2018, 7(2): 29-32.
[25]	Celona L, Mammana L. A multi-task CNN framework for driver face monitoring[C]// 2018 IEEE 8th Int’l Conf Consumer Electr, Berlin, 2018: 612-616.

疲劳等级	f_b min^-1	t_b s	t_c s	f_y min^-1	t_y s	f_h min^-1
不	> 15	< 0.3	< 2	< 1	< 0.5	< 3
轻度	10~15	0.3~0.5	2~3	1~2	0.5~2	3~6
中度	6~10	0.5~1	3~5	2~4	2~5	6~10
严重	< 6	> 1	> 5	> 4	> 5	> 10

疲劳等级	f_b min^-1	t_b s	t_c s	f_y min^-1	t_y s	f_h min^-1
不	> 15	< 0.3	< 2	< 1	< 0.5	< 3
轻度	10~15	0.3~0.5	2~3	1~2	0.5~2	3~6
中度	6~10	0.5~1	3~5	2~4	2~5	6~10
严重	< 6	> 1	> 5	> 4	> 5	> 10

算法模型	正常光照情况		低光照情况
算法模型	准确率/ %	识别时间/ ms	准确率/ %	识别时间/ ms
TVA	66	69	62	78
SVM	80	282	74	272
Ada-Boost	87	242	85	253
DT	75	206	77	224
KNN	78	185	70	181
CNN	91	125	90	132

算法模型	正常光照情况		低光照情况
算法模型	准确率/ %	识别时间/ ms	准确率/ %	识别时间/ ms
TVA	66	69	62	78
SVM	80	282	74	272
Ada-Boost	87	242	85	253
DT	75	206	77	224
KNN	78	185	70	181
CNN	91	125	90	132