A Simple Strategy to Provable Invariance via Orbit Mapping

Many applications require robustness, or ideally invariance, of neural networks to certain transformations of input data. Most commonly, this requirement is addressed by training data augmentation, using adversarial training, or defining network architectures that include the desired invariance by design. In this work, we propose a method to make network architectures provably invariant with respect to group actions by choosing one element from a (possibly continuous) orbit based on a fixed criterion. In a nutshell, we intend to 'undo' any possible transformation before feeding the data into the actual network. Further, we empirically analyze the properties of different approaches which incorporate invariance via training or architecture, and demonstrate the advantages of our method in terms of robustness and computational efficiency. In particular, we investigate the robustness with respect to rotations of images (which can hold up to discretization artifacts) as well as the provable orientation and scaling invariance of 3D point cloud classification.


page 1

page 2

page 3

page 4


Training or Architecture? How to Incorporate Invariance in Neural Networks

Many applications require the robustness, or ideally the invariance, of ...

Invariance vs. Robustness of Neural Networks

We study the performance of neural network models on random geometric tr...

Saliency Map Based Data Augmentation

Data augmentation is a commonly applied technique with two seemingly rel...

Deep invariant networks with differentiable augmentation layers

Designing learning systems which are invariant to certain data transform...

Malign Overfitting: Interpolation Can Provably Preclude Invariance

Learned classifiers should often possess certain invariance properties m...

Regularising for invariance to data augmentation improves supervised learning

Data augmentation is used in machine learning to make the classifier inv...

A Structural Approach to the Design of Domain Specific Neural Network Architectures

This is a master's thesis concerning the theoretical ideas of geometric ...

Please sign up or login with your details

Forgot password? Click here to reset