Why current rain denoising models fail on CycleGAN created rain images in autonomous driving

by   Michael Kranl, et al.

One of the main tasks of an autonomous agent in a vehicle is to correctly perceive its environment. Much of the data that needs to be processed is collected by optical sensors such as cameras. Unfortunately, the data collected in this way can be affected by a variety of factors, including environmental influences such as inclement weather conditions (e.g., rain). Such noisy data can cause autonomous agents to take wrong decisions with potentially fatal outcomes. This paper addresses the rain image challenge by two steps: First, rain is artificially added to a set of clear-weather condition images using a Generative Adversarial Network (GAN). This yields good/bad weather image pairs for training de-raining models. This artificial generation of rain images is sufficiently realistic as in 7 out of 10 cases, human test subjects believed the generated rain images to be real. In a second step, this paired good/bad weather image data is used to train two rain denoising models, one based primarily on a Convolutional Neural Network (CNN) and the other using a Vision Transformer. This rain de-noising step showed limited performance as the quality gain was only about 15 images as used in our study is likely due to current rain de-noising models being developed for simplistic rain overlay data. Our study shows that there is ample space for improvement of de-raining models in autonomous driving.


page 3

page 4

page 6


Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images

For safety of autonomous driving, vehicles need to be able to drive unde...

DeepRoad: GAN-based Metamorphic Autonomous Driving System Testing

While Deep Neural Networks (DNNs) have established the fundamentals of D...

WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models

The open road poses many challenges to autonomous perception, including ...

Safe Autonomous Driving in Adverse Weather: Sensor Evaluation and Performance Monitoring

The vehicle's perception sensors radar, lidar and camera, which must wor...

Task-Driven Deep Image Enhancement Network for Autonomous Driving in Bad Weather

Visual perception in autonomous driving is a crucial part of a vehicle t...

Let's Get Dirty: GAN Based Data Augmentation for Soiling and Adverse Weather Classification in Autonomous Driving

Cameras are getting more and more important in autonomous driving. Wide-...

RestoreX-AI: A Contrastive Approach towards Guiding Image Restoration via Explainable AI Systems

Modern applications such as self-driving cars and drones rely heavily up...

Please sign up or login with your details

Forgot password? Click here to reset