Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs

by   Furkan Ozcelik, et al.

Convolutional Neural Networks (CNN)-based approaches have shown promising results in pansharpening of satellite images in recent years. However, they still exhibit limitations in producing high-quality pansharpening outputs. To that end, we propose a new self-supervised learning framework, where we treat pansharpening as a colorization problem, which brings an entirely novel perspective and solution to the problem compared to existing methods that base their solution solely on producing a super-resolution version of the multispectral image. Whereas CNN-based methods provide a reduced resolution panchromatic image as input to their model along with reduced resolution multispectral images, hence learn to increase their resolution together, we instead provide the grayscale transformed multispectral image as input, and train our model to learn the colorization of the grayscale input. We further address the fixed downscale ratio assumption during training, which does not generalize well to the full-resolution scenario. We introduce a noise injection into the training by randomly varying the downsampling ratios. Those two critical changes, along with the addition of adversarial training in the proposed PanColorization Generative Adversarial Networks (PanColorGAN) framework, help overcome the spatial detail loss and blur problems that are observed in CNN-based pansharpening. The proposed approach outperforms the previous CNN-based and traditional methods as demonstrated in our experiments.


page 3

page 5

page 6

page 10

page 11

page 13

page 14

page 16


SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution

Single image super resolution (SISR) is to reconstruct a high resolution...

Learning a Single Convolutional Super-Resolution Network for Multiple Degradations

Recent years have witnessed the unprecedented success of deep convolutio...

Advances on CNN-based super-resolution of Sentinel-2 images

Thanks to their temporal-spatial coverage and free access, Sentinel-2 im...

Joint Face Super-Resolution and Deblurring Using a Generative Adversarial Network

Facial image super-resolution (SR) is an important preprocessing for fac...

Comparing Adversarial and Supervised Learning for Organs at Risk Segmentation in CT images

Organ at Risk (OAR) segmentation from CT scans is a key component of the...

Novel and Effective CNN-Based Binarization for Historically Degraded As-built Drawing Maps

Binarizing historically degraded as-built drawing (HDAD) maps is a new c...

Increasing diversity of omni-directional images generated from single image using cGAN based on MLPMixer

This paper proposes a novel approach to generating omni-directional imag...

Please sign up or login with your details

Forgot password? Click here to reset