Universal approximation property of invertible neural networks

by   Isao Ishikawa, et al.

Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning. However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry. The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs. We apply the framework to two representative classes of INNs, namely Coupling-Flow-based INNs (CF-INNs) and Neural Ordinary Differential Equations (NODEs), and elucidate their high representation power despite the restrictions on their architectures.


page 1

page 2

page 3

page 4


Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators

Invertible neural networks based on coupling flows (CF-INNs) have variou...

Universal Approximation Property of Neural Ordinary Differential Equations

Neural ordinary differential equations (NODEs) is an invertible neural n...

Distribution learning via neural differential equations: a nonparametric statistical perspective

Ordinary differential equations (ODEs), via their induced flow maps, pro...

Neural Oscillators are Universal

Coupled oscillators are being increasingly used as the basis of machine ...

Universal Approximation Property of Hamiltonian Deep Neural Networks

This paper investigates the universal approximation capabilities of Hami...

Asymptotic Properties of Neural Network Sieve Estimators

Neural networks are one of the most popularly used methods in machine le...

Universal Approximation by a Slim Network with Sparse Shortcut Connections

Over recent years, deep learning has become a mainstream method in machi...

Please sign up or login with your details

Forgot password? Click here to reset