Gotta Go Fast When Generating Data with Score-Based Models

Score-based (denoising diffusion) generative models have recently gained a lot of success in generating realistic and diverse data. These approaches define a forward diffusion process for transforming data to noise and generate data by reversing it (thereby going from noise to data). Unfortunately, current score-based models generate data very slowly due to the sheer number of score network evaluations required by numerical SDE solvers. In this work, we aim to accelerate this process by devising a more efficient SDE solver. Existing approaches rely on the Euler-Maruyama (EM) solver, which uses a fixed step size. We found that naively replacing it with other SDE solvers fares poorly - they either result in low-quality samples or become slower than EM. To get around this issue, we carefully devise an SDE solver with adaptive step sizes tailored to score-based generative models piece by piece. Our solver requires only two score function evaluations, rarely rejects samples, and leads to high-quality samples. Our approach generates data 2 to 10 times faster than EM while achieving better or equal sample quality. For high-resolution images, our method leads to significantly higher quality samples than all other methods tested. Our SDE solver has the benefit of requiring no step size tuning.


page 19

page 20

page 21

page 22

page 23

page 24


SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models

Diffusion Probabilistic Models (DPMs) have achieved considerable success...

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models

Diffusion probabilistic models (DPMs) have achieved impressive success i...

Post-training Quantization on Diffusion Models

Denoising diffusion (score-based) generative models have recently achiev...

GENIE: Higher-Order Denoising Diffusion Solvers

Denoising diffusion models (DDMs) have emerged as a powerful class of ge...

Generating symbolic music using diffusion models

Probabilistic Denoising Diffusion models have emerged as simple yet very...

SOS: Score-based Oversampling for Tabular Data

Score-based generative models (SGMs) are a recent breakthrough in genera...

How to train your neural ODE

Training neural ODEs on large datasets has not been tractable due to the...

Please sign up or login with your details

Forgot password? Click here to reset