Probabilistic Margins for Instance Reweighting in Adversarial Training

by   Qizhou Wang, et al.

Reweighting adversarial data during training has been recently shown to improve adversarial robustness, where data closer to the current decision boundaries are regarded as more critical and given larger weights. However, existing methods measuring the closeness are not very reliable: they are discrete and can take only a few values, and they are path-dependent, i.e., they may change given the same start and end points with different attack paths. In this paper, we propose three types of probabilistic margin (PM), which are continuous and path-independent, for measuring the aforementioned closeness and reweighting adversarial data. Specifically, a PM is defined as the difference between two estimated class-posterior probabilities, e.g., such the probability of the true label minus the probability of the most confusing label given some natural data. Though different PMs capture different geometric properties, all three PMs share a negative correlation with the vulnerability of data: data with larger/smaller PMs are safer/riskier and should have smaller/larger weights. Experiments demonstrate that PMs are reliable measurements and PM-based reweighting methods outperform state-of-the-art methods.


page 1

page 2

page 3

page 4


Local Reweighting for Adversarial Training

Instances-reweighted adversarial training (IRAT) can significantly boost...

Geometry-aware Instance-reweighted Adversarial Training

In adversarial machine learning, there was a common belief that robustne...

Switching One-Versus-the-Rest Loss to Increase the Margin of Logits for Adversarial Robustness

Defending deep neural networks against adversarial examples is a key cha...

A Context-Aware Approach for Textual Adversarial Attack through Probability Difference Guided Beam Search

Textual adversarial attacks expose the vulnerabilities of text classifie...

Doubly Robust Instance-Reweighted Adversarial Training

Assigning importance weights to adversarial data has achieved great succ...

Adversarial Robustness through the Lens of Causality

The adversarial vulnerability of deep neural networks has attracted sign...

Please sign up or login with your details

Forgot password? Click here to reset