Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling

by   Tong Che, et al.

We show that the sum of the implicit generator log-density log p_g of a GAN with the logit score of the discriminator defines an energy function which yields the true data density when the generator is imperfect but the discriminator is optimal, thus making it possible to improve on the typical generator (with implicit density p_g). To make that practical, we show that sampling from this modified density can be achieved by sampling in latent space according to an energy-based model induced by the sum of the latent prior log-density and the discriminator output score. This can be achieved by running a Langevin MCMC in latent space and then applying the generator function, which we call Discriminator Driven Latent Sampling (DDLS). We show that DDLS is highly efficient compared to previous methods which work in the high-dimensional pixel space and can be applied to improve on previously trained GANs of many types. We evaluate DDLS on both synthetic and real-world datasets qualitatively and quantitatively. On CIFAR-10, DDLS substantially improves the Inception Score of an off-the-shelf pre-trained SN-GAN <cit.> from 8.22 to 9.09 which is even comparable to the class-conditional BigGAN <cit.> model. This achieves a new state-of-the-art in unconditional image synthesis setting without introducing extra parameters or additional training.


page 7

page 8

page 13


Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions

We present a neural network architecture based upon the Autoencoder (AE)...

Discriminator optimal transport

Within a broad class of generative adversarial networks, we show that di...

Latent reweighting, an almost free improvement for GANs

Standard formulations of GANs, where a continuous function deforms a con...

Discriminator Rejection Sampling

We propose a rejection sampling scheme using the discriminator of a GAN ...

The Implicit Metropolis-Hastings Algorithm

Recent works propose using the discriminator of a GAN to filter out unre...

Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC

Learning energy-based model (EBM) requires MCMC sampling of the learned ...

OCT-GAN: Neural ODE-based Conditional Tabular GANs

Synthesizing tabular data is attracting much attention these days for va...

Please sign up or login with your details

Forgot password? Click here to reset