SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining

by   Shen Zheng, et al.

Deep learning algorithms have recently achieved promising deraining performances on both the natural and synthetic rainy datasets. As an essential low-level pre-processing stage, a deraining network should clear the rain streaks and preserve the fine semantic details. However, most existing methods only consider low-level image restoration. That limits their performances at high-level tasks requiring precise semantic information. To address this issue, in this paper, we present a segmentation-aware progressive network (SAPNet) based upon contrastive learning for single image deraining. We start our method with a lightweight derain network formed with progressive dilated units (PDU). The PDU can significantly expand the receptive field and characterize multi-scale rain streaks without the heavy computation on multi-scale images. A fundamental aspect of this work is an unsupervised background segmentation (UBS) network initialized with ImageNet and Gaussian weights. The UBS can faithfully preserve an image's semantic information and improve the generalization ability to unseen photos. Furthermore, we introduce a perceptual contrastive loss (PCL) and a learned perceptual image similarity loss (LPISL) to regulate model learning. By exploiting the rainy image and groundtruth as the negative and the positive sample in the VGG-16 latent space, we bridge the fine semantic details between the derained image and the groundtruth in a fully constrained manner. Comprehensive experiments on synthetic and real-world rainy images show our model surpasses top-performing methods and aids object detection and semantic segmentation with considerable efficacy. A Pytorch Implementation is available at


page 1

page 3

page 4

page 6

page 7

page 8


Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining

Rain streaks significantly decrease the visibility of captured images an...

Progressive Semantic-Aware Style Transformation for Blind Face Restoration

Face restoration is important in face image processing, and has been wid...

Deeply supervised neural network with short connections for retinal vessel segmentation

The condition of vessel of the human eye is a fundamental factor for the...

Contrastive Learning for Compact Single Image Dehazing

Single image dehazing is a challenging ill-posed problem due to the seve...

Contrastive Semantic-Guided Image Smoothing Network

Image smoothing is a fundamental low-level vision task that aims to pres...

A Causal Intervention Scheme for Semantic Segmentation of Quasi-periodic Cardiovascular Signals

Precise segmentation is a vital first step to analyze semantic informati...

CS-PCN: Context-Space Progressive Collaborative Network for Image Denoising

Currently, image-denoising methods based on deep learning cannot adequat...

Please sign up or login with your details

Forgot password? Click here to reset