Elucidating the solution space of extended reverse-time SDE for diffusion models

by   Qinpeng Cui, et al.

Diffusion models (DMs) demonstrate potent image generation capabilities in various generative modeling tasks. Nevertheless, their primary limitation lies in slow sampling speed, requiring hundreds or thousands of sequential function evaluations through large neural networks to generate high-quality images. Sampling from DMs can be seen as solving corresponding stochastic differential equations (SDEs) or ordinary differential equations (ODEs). In this work, we formulate the sampling process as an extended reverse-time SDE (ER SDE), unifying prior explorations into ODEs and SDEs. Leveraging the semi-linear structure of ER SDE solutions, we offer exact solutions and arbitrarily high-order approximate solutions for VP SDE and VE SDE, respectively. Based on the solution space of the ER SDE, we yield mathematical insights elucidating the superior performance of ODE solvers over SDE solvers in terms of fast sampling. Additionally, we unveil that VP SDE solvers stand on par with their VE SDE counterparts. Finally, we devise fast and training-free samplers, ER-SDE Solvers, elevating the efficiency of stochastic samplers to unprecedented levels. Experimental results demonstrate achieving 3.45 FID in 20 function evaluations and 2.24 FID in 50 function evaluations on the ImageNet 64×64 dataset.


page 1

page 2

page 25

page 26

page 27

page 28

page 29

page 33


DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

Diffusion probabilistic models (DPMs) are emerging powerful generative m...

SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models

A potent class of generative models known as Diffusion Probabilistic Mod...

Fast Sampling of Diffusion Models via Operator Learning

Diffusion models have found widespread adoption in various areas. Howeve...

On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximation

One popular diffusion-based sampling strategy attempts to solve the reve...

Accelerating Guided Diffusion Sampling with Splitting Numerical Methods

Guided diffusion is a technique for conditioning the output of a diffusi...

Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling

Efficient differential equation solvers have significantly reduced the s...

UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models

Diffusion probabilistic models (DPMs) have demonstrated a very promising...

Please sign up or login with your details

Forgot password? Click here to reset