From Denoising Diffusions to Denoising Markov Models

11/07/2022
by   Joe Benton, et al.
0

Denoising diffusions are state-of-the-art generative models which exhibit remarkable empirical performance and come with theoretical guarantees. The core idea of these models is to progressively transform the empirical data distribution into a simple Gaussian distribution by adding noise using a diffusion. We obtain new samples whose distribution is close to the data distribution by simulating a "denoising" diffusion approximating the time reversal of this "noising" diffusion. This denoising diffusion relies on approximations of the logarithmic derivatives of the noised data densities, known as scores, obtained using score matching. Such models can be easily extended to perform approximate posterior simulation in high-dimensional scenarios where one can only sample from the prior and simulate synthetic observations from the likelihood. These methods have been primarily developed for data on ℝ^d while extensions to more general spaces have been developed on a case-by-case basis. We propose here a general framework which not only unifies and generalizes this approach to a wide class of spaces but also leads to an original extension of score matching. We illustrate the resulting class of denoising Markov models on various applications.

READ FULL TEXT

page 13

page 37

research
02/27/2023

Denoising Diffusion Samplers

Denoising diffusion models are a popular class of generative models prov...
research
09/06/2017

On the exact relationship between the denoising function and the data distribution

We prove an exact relationship between the optimal denoising function an...
research
08/10/2022

Convergence of denoising diffusion models under the manifold hypothesis

Denoising diffusion models are a recent class of generative models exhib...
research
03/03/2023

Diffusion Models are Minimax Optimal Distribution Estimators

While efficient distribution learning is no doubt behind the groundbreak...
research
12/17/2021

Heavy-tailed denoising score matching

Score-based model research in the last few years has produced state of t...
research
05/05/2023

Iterative α-(de)Blending: a Minimalist Deterministic Diffusion Model

We derive a minimalist but powerful deterministic denoising-diffusion mo...
research
01/31/2020

Generative Modeling with Denoising Auto-Encoders and Langevin Sampling

We study convergence of a generative modeling method that first estimate...

Please sign up or login with your details

Forgot password? Click here to reset