D2C: Diffusion-Denoising Models for Few-shot Conditional Generation

by   Abhishek Sinha, et al.

Conditional generative models of high-dimensional images have many applications, but supervision signals from conditions to images can be expensive to acquire. This paper describes Diffusion-Decoding models with Contrastive representations (D2C), a paradigm for training unconditional variational autoencoders (VAEs) for few-shot conditional image generation. D2C uses a learned diffusion-based prior over the latent representations to improve generation and contrastive self-supervised learning to improve representation quality. D2C can adapt to novel generation tasks conditioned on labels or manipulation constraints, by learning from as few as 100 labeled examples. On conditional generation from new labels, D2C achieves superior performance over state-of-the-art VAEs and diffusion models. On conditional image manipulation, D2C generations are two orders of magnitude faster to produce over StyleGAN2 ones and are preferred by 50 study.


page 8

page 25

page 26

page 27

page 28

page 29

page 30

page 31


IC3D: Image-Conditioned 3D Diffusion for Shape Generation

In the last years, Denoising Diffusion Probabilistic Models (DDPMs) obta...

Why Are Conditional Generative Models Better Than Unconditional Ones?

Extensive empirical evidence demonstrates that conditional generative mo...

Conditional Generation from Unconditional Diffusion Models using Denoiser Representations

Denoising diffusion models have gained popularity as a generative modeli...

Hierarchical Text-Conditional Image Generation with CLIP Latents

Contrastive models like CLIP have been shown to learn robust representat...

Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation

We aim to build image generation models that generalize to new domains f...

DifFSS: Diffusion Model for Few-Shot Semantic Segmentation

Diffusion models have demonstrated excellent performance in image genera...

Discrete Contrastive Diffusion for Cross-Modal and Conditional Generation

Diffusion probabilistic models (DPMs) have become a popular approach to ...

Please sign up or login with your details

Forgot password? Click here to reset