Diverse super-resolution with pretrained deep hiererarchical VAEs

by   Jean Prost, et al.

Image super-resolution is a one-to-many problem, but most deep-learning based methods only provide one single solution to this problem. In this work, we tackle the problem of diverse super-resolution by reusing VD-VAE, a state-of-the art variational autoencoder (VAE). We find that the hierarchical latent representation learned by VD-VAE naturally separates the image low-frequency information, encoded in the latent groups at the top of the hierarchy, from the image high-frequency details, determined by the latent groups at the bottom of the latent hierarchy. Starting from this observation, we design a super-resolution model exploiting the specific structure of VD-VAE latent space. Specifically, we train an encoder to encode low-resolution images in the subset of VD-VAE latent space encoding the low-frequency information, and we combine this encoder with VD-VAE generative model to sample diverse super-resolved version of a low-resolution input. We demonstrate the ability of our method to generate diverse solutions to the super-resolution problem on face super-resolution with upsampling factors x4, x8, and x16.


page 2

page 6

page 12


Variational AutoEncoder for Reference based Image Super-Resolution

In this paper, we propose a novel reference based image super-resolution...

Soft-IntroVAE for Continuous Latent space Image Super-Resolution

Continuous image super-resolution (SR) recently receives a lot of attent...

Progressive VAE Training on Highly Sparse and Imbalanced Data

In this paper, we present a novel approach for training a Variational Au...

Interpretable Super-Resolution via a Learned Time-Series Representation

We develop an interpretable and learnable Wigner-Ville distribution that...

WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolution

Audio super-resolution is the task of constructing a high-resolution (HR...

Lessons Learned Report: Super-Resolution for Detection Tasks in Engineering Problem-Solving

We describe the lessons learned from targeting agricultural detection pr...

Deep Reparametrization of Multi-Frame Super-Resolution and Denoising

We propose a deep reparametrization of the maximum a posteriori formulat...

Please sign up or login with your details

Forgot password? Click here to reset