Dynamic Dual-Output Diffusion Models

by   Yaniv Benny, et al.

Iterative denoising-based generation, also known as denoising diffusion models, has recently been shown to be comparable in quality to other classes of generative models, and even surpass them. Including, in particular, Generative Adversarial Networks, which are currently the state of the art in many sub-tasks of image generation. However, a major drawback of this method is that it requires hundreds of iterations to produce a competitive result. Recent works have proposed solutions that allow for faster generation with fewer iterations, but the image quality gradually deteriorates with increasingly fewer iterations being applied during generation. In this paper, we reveal some of the causes that affect the generation quality of diffusion models, especially when sampling with few iterations, and come up with a simple, yet effective, solution to mitigate them. We consider two opposite equations for the iterative denoising, the first predicts the applied noise, and the second predicts the image directly. Our solution takes the two options and learns to dynamically alternate between them through the denoising process. Our proposed solution is general and can be applied to any existing diffusion model. As we show, when applied to various SOTA architectures, our solution immediately improves their generation quality, with negligible added complexity and parameters. We experiment on multiple datasets and configurations and run an extensive ablation study to support these findings.


page 6

page 7

page 8

page 12

page 13

page 14


Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed

Iterative generative models, such as noise conditional score networks an...

Denoising Diffusion Implicit Models

Denoising diffusion probabilistic models (DDPMs) have achieved high qual...

Parallel Sampling of Diffusion Models

Diffusion models are powerful generative models but suffer from slow sam...

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

A wide variety of deep generative models has been developed in the past ...

Preconditioned Score-based Generative Models

Score-based generative models (SGMs) have recently emerged as a promisin...

VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Models

Diffusion Models (DMs) are state-of-the-art generative models that learn...

Markup-to-Image Diffusion Models with Scheduled Sampling

Building on recent advances in image generation, we present a fully data...

Please sign up or login with your details

Forgot password? Click here to reset