On Transferability of Driver Observation Models from Simulated to Real Environments in Autonomous Cars

by   Walter Morales Alvarez, et al.

For driver observation frameworks, clean datasets collected in controlled simulated environments often serve as the initial training ground. Yet, when deployed under real driving conditions, such simulator-trained models quickly face the problem of distributional shifts brought about by changing illumination, car model, variations in subject appearances, sensor discrepancies, and other environmental alterations. This paper investigates the viability of transferring video-based driver observation models from simulation to real-world scenarios in autonomous vehicles, given the frequent use of simulation data in this domain due to safety issues. To achieve this, we record a dataset featuring actual autonomous driving conditions and involving seven participants engaged in highly distracting secondary activities. To enable direct SIM to REAL transfer, our dataset was designed in accordance with an existing large-scale simulator dataset used as the training source. We utilize the Inflated 3D ConvNet (I3D) model, a popular choice for driver observation, with Gradient-weighted Class Activation Mapping (Grad-CAM) for detailed analysis of model decision-making. Though the simulator-based model clearly surpasses the random baseline, its recognition quality diminishes, with average accuracy dropping from 85.7 46.6 This underscores the challenges of model transferability, facilitating our research of more robust driver observation systems capable of dealing with real driving conditions.


page 1

page 2

page 4

page 5

page 6


Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Data

With increasing automation in passenger vehicles, the study of safe and ...

Driver State and Behavior Detection Through Smart Wearables

Integrating driver, in-cabin, and outside environment's contextual cues ...

Enhancing SUMO simulator for simulation based testing and validation of autonomous vehicles

Current autonomous vehicle (AV) simulators are built to provide large-sc...

Human-Vehicle Cooperation on Prediction-Level: Enhancing Automated Driving with Human Foresight

To maximize safety and driving comfort, autonomous driving systems can b...

Automated Failure-Mode Clustering and Labeling for Informed Car-To-Driver Handover in Autonomous Vehicles

The car-to-driver handover is a critically important component of safe a...

Conditional Latent Block Model: a Multivariate Time Series Clustering Approach for Autonomous Driving Validation

Autonomous driving systems validation remains one of the biggest challen...

Please sign up or login with your details

Forgot password? Click here to reset