Driver Fatigue Detection Based on Residual Channel Attention Network and Head Pose Estimation

Abstract

Driver fatigue is the culprit of most traffic accidents. Visual technology can intuitively judge whether the driver is in the state of fatigue. A driver fatigue detection system based on the residual channel attention network (RCAN) and head pose estimation is proposed. In the proposed system, Retinaface is employed for face location and outputs five face landmarks. Then the RCAN is proposed to classify the state of eyes and the mouth. The RCAN includes a channel attention module, which can adaptively extract key feature vectors from the feature map, which significantly improves the classification accuracy of the RCAN. In the self-built dataset, the classification accuracy of the eye state of the RCAN reaches 98.962% and that of the mouth state reaches 98.561%, exceeding other classical convolutional neural networks. The percentage of eyelid closure over the pupil over time (PERCLOS) and the mouth opening degree (POM) are used for fatigue detection based on the state of eyes and the mouth. In addition, this article proposes to use a Perspective-n-Point (PnP) method to estimate the head pose as an essential supplement for driving fatigue detection and proposes over-angle to evaluate whether the head pose is excessively deflected. On the whole, the proposed driver fatigue system integrates 3D head pose estimation and fatigue detection based on deep learning. This system is evaluated by the four datasets and shows success of the proposed method with their high performance.