Deep Equilibrium Approaches to Diffusion Models

by   Ashwini Pokle, et al.
Carnegie Mellon University

Diffusion-based generative models are extremely effective in generating high-quality images, with generated samples often surpassing the quality of those produced by other models under several metrics. One distinguishing feature of these models, however, is that they typically require long sampling chains to produce high-fidelity images. This presents a challenge not only from the lenses of sampling time, but also from the inherent difficulty in backpropagating through these chains in order to accomplish tasks such as model inversion, i.e. approximately finding latent states that generate known images. In this paper, we look at diffusion models through a different perspective, that of a (deep) equilibrium (DEQ) fixed point model. Specifically, we extend the recent denoising diffusion implicit model (DDIM; Song et al. 2020), and model the entire sampling chain as a joint, multivariate fixed point system. This setup provides an elegant unification of diffusion and equilibrium models, and shows benefits in 1) single image sampling, as it replaces the fully-serial typical sampling process with a parallel one; and 2) model inversion, where we can leverage fast gradients in the DEQ setting to much more quickly find the noise that generates a given image. The approach is also orthogonal and thus complementary to other methods used to reduce the sampling time, or improve model inversion. We demonstrate our method's strong performance across several datasets, including CIFAR10, CelebA, and LSUN Bedrooms and Churches.


page 9

page 10

page 18

page 21

page 24

page 25

page 26

page 27


Generating High Fidelity Data from Low-density Regions using Diffusion Models

Our work focuses on addressing sample deficiency from low-density region...

Spontaneous symmetry breaking in generative diffusion models

Generative diffusion models have recently emerged as a leading approach ...

Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium

Regularized inversion methods for image reconstruction are used widely d...

A prior regularized full waveform inversion using generative diffusion models

Full waveform inversion (FWI) has the potential to provide high-resoluti...

EDICT: Exact Diffusion Inversion via Coupled Transformations

Finding an initial noise vector that produces an input image when fed in...

Training on Thin Air: Improve Image Classification with Generated Data

Acquiring high-quality data for training discriminative models is a cruc...

Joint inference and input optimization in equilibrium networks

Many tasks in deep learning involve optimizing over the inputs to a netw...

Please sign up or login with your details

Forgot password? Click here to reset