Self Normalizing Flows

by   T. Anderson Keller, et al.

Efficient gradient computation of the Jacobian determinant term is a core problem of the normalizing flow framework. Thus, most proposed flow models either restrict to a function class with easy evaluation of the Jacobian determinant, or an efficient estimator thereof. However, these restrictions limit the performance of such density models, frequently requiring significant depth to reach desired performance levels. In this work, we propose Self Normalizing Flows, a flexible framework for training normalizing flows by replacing expensive terms in the gradient by learned approximate inverses at each layer. This reduces the computational complexity of each layer's exact update from π’ͺ(D^3) to π’ͺ(D^2), allowing for the training of flow architectures which were otherwise computationally infeasible, while also providing efficient sampling. We show experimentally that such models are remarkably stable and optimize to similar data likelihood values as their exact gradient counterparts, while surpassing the performance of their functionally constrained counterparts.

βˆ™ 12/13/2021

ELF: Exact-Lipschitz Based Universal Density Approximator Flow

Normalizing flows have grown more popular over the last few years; howev...
βˆ™ 02/22/2023

From Optimization to Sampling Through Gradient Flows

This article overviews how gradient flows, and discretizations thereof, ...
βˆ™ 02/17/2020

Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models

In this work, we propose a new family of generative flows on an augmente...
βˆ™ 03/22/2022

VQ-Flows: Vector Quantized Local Normalizing Flows

Normalizing flows provide an elegant approach to generative modeling tha...
βˆ™ 02/27/2020

Woodbury Transformations for Deep Generative Flows

Normalizing flows are deep generative models that allow efficient likeli...
βˆ™ 03/04/2020

Gaussianization Flows

Iterative Gaussianization is a fixed-point iteration procedure that can ...
βˆ™ 09/02/2019

Randomized methods to characterize large-scale vortical flow network

We demonstrate the effective use of randomized methods for linear algebr...

Please sign up or login with your details

Forgot password? Click here to reset