On the expressivity of bi-Lipschitz normalizing flows

by   Alexandre Verine, et al.

An invertible function is bi-Lipschitz if both the function and its inverse have bounded Lipschitz constants. Nowadays, most Normalizing Flows are bi-Lipschitz by design or by training to limit numerical errors (among other things). In this paper, we discuss the expressivity of bi-Lipschitz Normalizing Flows and identify several target distributions that are difficult to approximate using such models. Then, we characterize the expressivity of bi-Lipschitz Normalizing Flows by giving several lower bounds on the Total Variation distance between these particularly unfavorable distributions and their best possible approximation. Finally, we discuss potential remedies which include using more complex latent distributions.


page 1

page 2

page 3

page 4


Lipschitz bijections between boolean functions

We answer four questions from a recent paper of Rao and Shinkar on Lipsc...

Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective

ResNets constrained to be bi-Lipschitz, that is, approximately distance ...

On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networks

Invertible neural networks (INNs) represent an important class of deep n...

ELF: Exact-Lipschitz Based Universal Density Approximator Flow

Normalizing flows have grown more popular over the last few years; howev...

Invertible Monotone Operators for Normalizing Flows

Normalizing flows model probability distributions by learning invertible...

Nonlinear Dimension Reduction via Outer Bi-Lipschitz Extensions

We introduce and study the notion of an outer bi-Lipschitz extension of ...


We introduce Invertible Dense Networks (i-DenseNets), a more parameter e...

Please sign up or login with your details

Forgot password? Click here to reset