Interpolation for Robust Learning: Data Augmentation on Geodesics

by   Jiacheng Zhu, et al.

We propose to study and promote the robustness of a model as per its performance through the interpolation of training data distributions. Specifically, (1) we augment the data by finding the worst-case Wasserstein barycenter on the geodesic connecting subpopulation distributions of different categories. (2) We regularize the model for smoother performance on the continuous geodesic path connecting subpopulation distributions. (3) Additionally, we provide a theoretical guarantee of robustness improvement and investigate how the geodesic location and the sample size contribute, respectively. Experimental validations of the proposed strategy on four datasets, including CIFAR-100 and ImageNet, establish the efficacy of our method, e.g., our method improves the baselines' certifiable robustness on CIFAR10 up to 7.7%, with 16.8% on empirical robustness on CIFAR-100. Our work provides a new perspective of model robustness through the lens of Wasserstein geodesic-based interpolation with a practical off-the-shelf strategy that can be combined with existing robust training methods.


page 1

page 2

page 3

page 4


CEB Improves Model Robustness

We demonstrate that the Conditional Entropy Bottleneck (CEB) can improve...

k-Mixup Regularization for Deep Learning via Optimal Transport

Mixup is a popular regularization technique for training deep neural net...

Adversarially Optimized Mixup for Robust Classification

Mixup is a procedure for data augmentation that trains networks to make ...

Wasserstein Distributional Robustness and Regularization in Statistical Learning

A central question in statistical learning is to design algorithms that ...

Does Data Augmentation Benefit from Split BatchNorms

Data augmentation has emerged as a powerful technique for improving the ...

Data Augmentation with Manifold Barycenters

The training of Generative Adversarial Networks (GANs) requires a large ...

Connecting and Comparing Language Model Interpolation Techniques

In this work, we uncover a theoretical connection between two language m...

Please sign up or login with your details

Forgot password? Click here to reset