Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples

by   Vikash Sehwag, et al.
Princeton University

A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where training and test inputs follow a pre-specified distribution. However, real-world implementations of deep learning applications, such as autonomous driving and content classification are likely to operate in the open-world environment. In this paper, we demonstrate the success of open-world evasion attacks, where adversarial examples are generated from out-of-distribution inputs (OOD adversarial examples). In our study, we use 11 state-of-the-art neural network models trained on 3 image datasets of varying complexity. We first demonstrate that state-of-the-art detectors for out-of-distribution data are not robust against OOD adversarial examples. We then consider 5 known defenses for adversarial examples, including state-of-the-art robust training methods, and show that against these defenses, OOD adversarial examples can achieve up to 4× higher target success rates compared to adversarial examples generated from in-distribution data. We also take a quantitative look at how open-world evasion attacks may affect real-world systems. Finally, we present the first steps towards a robust open-world machine learning system.


page 2

page 12


Are adversarial examples inevitable?

A wide range of defenses have been proposed to harden neural networks ag...

A Critical Evaluation of Open-World Machine Learning

Open-world machine learning (ML) combines closed-world models trained on...

Synthesizing Robust Adversarial Examples

Neural network-based classifiers parallel or exceed human-level accuracy...

Rethinking Machine Learning Robustness via its Link with the Out-of-Distribution Problem

Despite multiple efforts made towards robust machine learning (ML) model...

Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verification

The phenomenon of Adversarial Examples is attracting increasing interest...

Targeted Deep Learning: Framework, Methods, and Applications

Deep learning systems are typically designed to perform for a wide range...

Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them

Making classifiers robust to adversarial examples is hard. Thus, many de...

Please sign up or login with your details

Forgot password? Click here to reset