Prior-Guided Adversarial Initialization for Fast Adversarial Training

by   Xiaojun Jia, et al.
University of the Chinese Academy of Sciences
Beihang University
The Chinese University of Hong Kong, Shenzhen

Fast adversarial training (FAT) effectively improves the efficiency of standard adversarial training (SAT). However, initial FAT encounters catastrophic overfitting, i.e.,the robust accuracy against adversarial attacks suddenly and dramatically decreases. Though several FAT variants spare no effort to prevent overfitting, they sacrifice much calculation cost. In this paper, we explore the difference between the training processes of SAT and FAT and observe that the attack success rate of adversarial examples (AEs) of FAT gets worse gradually in the late training stage, resulting in overfitting. The AEs are generated by the fast gradient sign method (FGSM) with a zero or random initialization. Based on the observation, we propose a prior-guided FGSM initialization method to avoid overfitting after investigating several initialization strategies, improving the quality of the AEs during the whole training process. The initialization is formed by leveraging historically generated AEs without additional calculation cost. We further provide a theoretical analysis for the proposed initialization method. We also propose a simple yet effective regularizer based on the prior-guided initialization,i.e., the currently generated perturbation should not deviate too much from the prior-guided initialization. The regularizer adopts both historical and current adversarial perturbations to guide the model learning. Evaluations on four datasets demonstrate that the proposed method can prevent catastrophic overfitting and outperform state-of-the-art FAT methods. The code is released at


page 1

page 2

page 3

page 4


Improving Fast Adversarial Training with Prior-Guided Knowledge

Fast adversarial training (FAT) is an efficient method to improve robust...

Boosting Fast Adversarial Training with Learnable Adversarial Initialization

Adversarial training (AT) has been demonstrated to be effective in impro...

Fast Adversarial Training with Smooth Convergence

Fast adversarial training (FAT) is beneficial for improving the adversar...

Bag of Tricks for FGSM Adversarial Training

Adversarial training (AT) with samples generated by Fast Gradient Sign M...

Fast Adversarial Training with Adaptive Step Size

While adversarial training and its variants have shown to be the most ef...

Fast is better than free: Revisiting adversarial training

Adversarial training, a method for learning robust deep networks, is typ...

Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attacks

Adversarial training (AT) with imperfect supervision is significant but ...

Please sign up or login with your details

Forgot password? Click here to reset