Training a Better Loss Function for Image Restoration

by   Aamir Mustafa, et al.

Central to the application of neural networks in image restoration problems, such as single image super resolution, is the choice of a loss function that encourages natural and perceptually pleasing results. A popular choice for a loss function is a pre-trained network, such as VGG and LPIPS, which is used as a feature extractor for computing the difference between restored and reference images. However, such an approach has multiple drawbacks: it is computationally expensive, requires regularization and hyper-parameter tuning, and involves a large network trained on an unrelated task. In this work, we explore the question of what makes a good loss function for an image restoration task. First, we observe that a single natural image is sufficient to train a lightweight feature extractor that outperforms state-of-the-art loss functions in single image super resolution, denoising, and JPEG artefact removal. We propose a novel Multi-Scale Discriminative Feature (MDF) loss comprising a series of discriminators, trained to penalize errors introduced by a generator. Second, we show that an effective loss function does not have to be a good predictor of perceived image quality, but instead needs to be specialized in identifying the distortions for a given restoration method.


page 4

page 6

page 8

page 9

page 10

page 11

page 12


Unlocking Masked Autoencoders as Loss Function for Image and Video Restoration

Image and video restoration has achieved a remarkable leap with the adve...

Generative Collaborative Networks for Single Image Super-Resolution

A common issue of deep neural networks-based methods for the problem of ...

Image quality measurements and denoising using Fourier Ring Correlations

Image quality is a nebulous concept with different meanings to different...

Random Weights Networks Work as Loss Prior Constraint for Image Restoration

In this paper, orthogonal to the existing data and model studies, we ins...

Feature Forwarding for Efficient Single Image Dehazing

Haze degrades content and obscures information of images, which can nega...

Learning the Loss Functions in a Discriminative Space for Video Restoration

With more advanced deep network architectures and learning schemes such ...

New Techniques for Preserving Global Structure and Denoising with Low Information Loss in Single-Image Super-Resolution

This work identifies and addresses two important technical challenges in...

Please sign up or login with your details

Forgot password? Click here to reset