GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks

by   Salah Ghamizi, et al.

While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we propose Guided Adversarial Training (GAT), a novel adversarial training technique that exploits auxiliary tasks under a limited set of training data. Our approach extends single-task models into multi-task models during the min-max optimization of adversarial training, and drives the loss optimization with a regularization of the gradient curvature across multiple tasks. GAT leverages two types of auxiliary tasks: self-supervised tasks, where the labels are generated automatically, and domain-knowledge tasks, where human experts provide additional labels. Experimentally, under limited data, GAT increases the robust accuracy on CIFAR-10 up to four times (from 11 accuracy) and the robust AUC of CheXpert medical imaging dataset from 50% to 83%. On the full CIFAR-10 dataset, GAT outperforms eight state-of-the-art adversarial training strategies. Our large study across five datasets and six tasks demonstrates that task augmentation is an efficient alternative to data augmentation, and can be key to achieving both clean and robust performances.


page 1

page 2

page 3

page 4


Fooling Adversarial Training with Inducing Noise

Adversarial training is widely believed to be a reliable approach to imp...

Robust Machine Comprehension Models via Adversarial Training

It is shown that many published models for the Stanford Question Answeri...

Enabling Data Diversity: Efficient Automatic Augmentation via Regularized Adversarial Training

Data augmentation has proved extremely useful by increasing training dat...

One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks

Unlearnable examples (ULEs) aim to protect data from unauthorized usage ...

A Study on FGSM Adversarial Training for Neural Retrieval

Neural retrieval models have acquired significant effectiveness gains ov...

Semi-supervised Semantics-guided Adversarial Training for Trajectory Prediction

Predicting the trajectories of surrounding objects is a critical task in...

Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization

Many previous proposals for adversarial training of deep neural nets hav...

Please sign up or login with your details

Forgot password? Click here to reset