Real-time monitoring of driver drowsiness on mobile platforms using 3D neural networks

by   Jasper S. Wijnands, et al.

Driver drowsiness increases crash risk, leading to substantial road trauma each year. Drowsiness detection methods have received considerable attention, but few studies have investigated the implementation of a detection approach on a mobile phone. Phone applications reduce the need for specialised hardware and hence, enable a cost-effective roll-out of the technology across the driving population. While it has been shown that three-dimensional (3D) operations are more suitable for spatiotemporal feature learning, current methods for drowsiness detection commonly use frame-based, multi-step approaches. However, computationally expensive techniques that achieve superior results on action recognition benchmarks (e.g. 3D convolutions, optical flow extraction) create bottlenecks for real-time, safety-critical applications on mobile devices. Here, we show how depthwise separable 3D convolutions, combined with an early fusion of spatial and temporal information, can achieve a balance between high prediction accuracy and real-time inference requirements. In particular, increased accuracy is achieved when assessment requires motion information, for example, when sunglasses conceal the eyes. Further, a custom TensorFlow-based smartphone application shows the true impact of various approaches on inference times and demonstrates the effectiveness of real-time monitoring based on out-of-sample data to alert a drowsy driver. Our model is pre-trained on ImageNet and Kinetics and fine-tuned on a publicly available Driver Drowsiness Detection dataset. Fine-tuning on large naturalistic driving datasets could further improve accuracy to obtain robust in-vehicle performance. Overall, our research is a step towards practical deep learning applications, potentially preventing micro-sleeps and reducing road trauma.


Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach

Many road accidents occur due to distracted drivers. Today, driver monit...

Towards Real-Time Action Recognition on Mobile Devices Using Deep Models

Action recognition is a vital task in computer vision, and many methods ...

Vision Transformers and YoloV5 based Driver Drowsiness Detection Framework

Human drivers have distinct driving techniques, knowledge, and sentiment...

Robust Two-Stream Multi-Feature Network for Driver Drowsiness Detection

Drowsiness driving is a major cause of traffic accidents and thus numero...

Identification of Driver Phone Usage Violations via State-of-the-Art Object Detection with Tracking

The use of mobiles phones when driving have been a major factor when it ...

A Computer Vision-Based Approach for Driver Distraction Recognition using Deep Learning and Genetic Algorithm Based Ensemble

As the proportion of road accidents increases each year, driver distract...

SFF-DA: Sptialtemporal Feature Fusion for Detecting Anxiety Nonintrusively

Early detection of anxiety disorders is essential to reduce the sufferin...

Please sign up or login with your details

Forgot password? Click here to reset