Score-based Diffusion Models in Function Space

by   Jae Hyun Lim, et al.
Université de Montréal
Montréal Institute of Learning Algorithms
California Institute of Technology

Diffusion models have recently emerged as a powerful framework for generative modeling. They consist of a forward process that perturbs input data with Gaussian white noise and a reverse process that learns a score function to generate samples by denoising. Despite their tremendous success, they are mostly formulated on finite-dimensional spaces, e.g. Euclidean, limiting their applications to many domains where the data has a functional form such as in scientific computing and 3D geometric data analysis. In this work, we introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. In DDOs, the forward process perturbs input functions gradually using a Gaussian process. The generative process is formulated by integrating a function-valued Langevin dynamic. Our approach requires an appropriate notion of the score for the perturbed data distribution, which we obtain by generalizing denoising score matching to function spaces that can be infinite-dimensional. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost that is independent of the data resolution. We theoretically and numerically verify the applicability of our approach on a set of problems, including generating solutions to the Navier-Stokes equation viewed as the push-forward distribution of forcings from a Gaussian Random Field (GRF).


page 8

page 19

page 22

page 23


Blackout Diffusion: Generative Diffusion Models in Discrete-State Spaces

Typical generative diffusion models rely on a Gaussian diffusion process...

Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models for Image Generation

Score-based diffusion models (SBDM) have recently emerged as state-of-th...

Geometric Neural Diffusion Processes

Denoising diffusion models have proven to be a flexible and effective pa...

Regularizing Score-based Models with Score Fokker-Planck Equations

Score-based generative models learn a family of noise-conditional score ...

Functional Flow Matching

In this work, we propose Functional Flow Matching (FFM), a function-spac...

Using Intermediate Forward Iterates for Intermediate Generator Optimization

Score-based models have recently been introduced as a richer framework t...

Iterative α-(de)Blending: a Minimalist Deterministic Diffusion Model

We derive a minimalist but powerful deterministic denoising-diffusion mo...

Please sign up or login with your details

Forgot password? Click here to reset