Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework

by   Jie Chen, et al.

Rain removal is important for improving the robustness of outdoor vision based systems. Current rain removal methods show limitations either for complex dynamic scenes shot from fast moving cameras, or under torrential rain fall with opaque occlusions. We propose a novel derain algorithm, which applies superpixel (SP) segmentation to decompose the scene into depth consistent units. Alignment of scene contents are done at the SP level, which proves to be robust towards rain occlusion and fast camera motion. Two alignment output tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match tensor, provide informative clues for rain streak location and occluded background contents to generate an intermediate derain output. These tensors will be subsequently prepared as input features for a convolutional neural network to restore high frequency details to the intermediate output for compensation of mis-alignment blur. Extensive evaluations show that up to 5 dB reconstruction PSNR advantage is achieved over state-of-the-art methods. Visual inspection shows that much cleaner rain removal is achieved especially for highly dynamic scenes with heavy and opaque rainfall from a fast moving camera.


page 1

page 4

page 5

page 6

page 7


Robust Video Content Alignment and Compensation for Clear Vision Through the Rain

Outdoor vision-based systems suffer from atmospheric turbulences, and ra...

Deep learning based fence segmentation and removal from an image using a video sequence

Conventional approaches to image de-fencing use multiple adjacent frames...

Temporally coherent 4D reconstruction of complex dynamic scenes

This paper presents an approach for reconstruction of 4D temporally cohe...

Robust Consistent Video Depth Estimation

We present an algorithm for estimating consistent dense depth maps and c...

Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views

The extraction of a clean background image by removing foreground occlus...

Deep Learning for Robust Motion Segmentation with Non-Static Cameras

This work proposes a new end-to-end DCNN based approach for motion segme...

Video Rain/Snow Removal by Transformed Online Multiscale Convolutional Sparse Coding

Video rain/snow removal from surveillance videos is an important task in...

Please sign up or login with your details

Forgot password? Click here to reset