FatigueAlert: A real-time fatigue detection system using hybrid features and Pre-train mCNN model

摘要

Several computer-vision based applications are developed to detect of driver fatigue (DDF) and to decrease road accidents in a real-time environment. Those DDF systems were more focused on extracting visual-features. However, it is very much difficult to extract visual-features for defining PERCLOS measure due to different factors such as night-time driving, head is not centered-aligned and occlusion of faces. Due to these reasons, it is very much difficult to detect driver eyes, mouth and ears. As a result, some authors suggested using non-visual features combined with visual features to get accurate results. Accordingly, a hybrid and novel DDF system is developed in this paper by combining both visual and non-visual features through multi-cams stream approach and electrocardiography (ECG) sensors to measure heart rate variability (HRV). Those ECG sensors are mounted on driver’s steering. This DDF system is known as FatigueAlert and developed through deep architecture especially transfer-learning method. The proposed FatigueAlert system pre-trained many convolutional neural network (mCNN) models on different driver’s eyes, ears and mouths datasets. Three online datasets such as closed eyes in the wild (CEW), yawing dataset (YAWDD) and Columbia gaze dataset (CAVE-DB) were utilized to train and evaluate the proposed FatigueAlert system. On average, the FatigueAlert DDF system achieved 93.4% detection accuracy on different real-time driver’s datasets. To perform comparisons, different deep-learning models were used to compare with proposed pre-trained mCNN multi-layer architecture model. The obtained results indicate that the FatigueAlert system is outperformed compared to other state-of-the-art DDF systems.