Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling

by   Qinsheng Zhang, et al.

Efficient differential equation solvers have significantly reduced the sampling time of diffusion models (DMs) while retaining high sampling quality. Among these solvers, exponential integrators (EI) have gained prominence by demonstrating state-of-the-art performance. However, existing high-order EI-based sampling algorithms rely on degenerate EI solvers, resulting in inferior error bounds and reduced accuracy in contrast to the theoretically anticipated results under optimal settings. This situation makes the sampling quality extremely vulnerable to seemingly innocuous design choices such as timestep schedules. For example, an inefficient timestep scheduler might necessitate twice the number of steps to achieve a quality comparable to that obtained through carefully optimized timesteps. To address this issue, we reevaluate the design of high-order differential solvers for DMs. Through a thorough order analysis, we reveal that the degeneration of existing high-order EI solvers can be attributed to the absence of essential order conditions. By reformulating the differential equations in DMs and capitalizing on the theory of exponential integrators, we propose refined EI solvers that fulfill all the order conditions, which we designate as Refined Exponential Solver (RES). Utilizing these improved solvers, RES exhibits more favorable error bounds theoretically and achieves superior sampling efficiency and stability in practical applications. For instance, a simple switch from the single-step DPM-Solver++ to our order-satisfied RES solver when Number of Function Evaluations (NFE) =9, results in a reduction of numerical defects by 25.2% and FID improvement of 25.4% (16.77 vs 12.51) on a pre-trained ImageNet diffusion model.


page 23

page 24

page 25

page 26

page 27


SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models

Diffusion Probabilistic Models (DPMs) have achieved considerable success...

SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models

A potent class of generative models known as Diffusion Probabilistic Mod...

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models

Diffusion probabilistic models (DPMs) have achieved impressive success i...

Diffusion Sampling with Momentum for Mitigating Divergence Artifacts

Despite the remarkable success of diffusion models in image generation, ...

Elucidating the solution space of extended reverse-time SDE for diffusion models

Diffusion models (DMs) demonstrate potent image generation capabilities ...

A high order differential equation based wall distance solver

A computationally efficient high-order solver is developed to compute th...

On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximation

One popular diffusion-based sampling strategy attempts to solve the reve...

Please sign up or login with your details

Forgot password? Click here to reset