Robustness to Adversarial Perturbations in Learning from Incomplete Data

by   Amir Najafi, et al.
Sharif Accelerator
Preferred Infrastructure

What is the role of unlabeled data in an inference problem, when the presumed underlying distribution is adversarially perturbed? To provide a concrete answer to this question, this paper unifies two major learning frameworks: Semi-Supervised Learning (SSL) and Distributionally Robust Learning (DRL). We develop a generalization theory for our framework based on a number of novel complexity measures, such as an adversarial extension of Rademacher complexity and its semi-supervised analogue. Moreover, our analysis is able to quantify the role of unlabeled data in the generalization under a more general condition compared to the existing theoretical works in SSL. Based on our framework, we also present a hybrid of DRL and EM algorithms that has a guaranteed convergence rate. When implemented with deep neural networks, our method shows a comparable performance to those of the state-of-the-art on a number of real-world benchmark datasets.


Semi-Supervised Learning: the Case When Unlabeled Data is Equally Useful

Semi-supervised learning algorithms attempt to take advantage of relativ...

Adversarially Robust Generalization Just Requires More Unlabeled Data

Neural network robustness has recently been highlighted by the existence...

How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis

Self-training, a semi-supervised learning algorithm, leverages a large a...

Robust Deep Semi-Supervised Learning: A Brief Introduction

Semi-supervised learning (SSL) is the branch of machine learning that ai...

Incorporating Unlabeled Data into Distributionally Robust Learning

We study a robust alternative to empirical risk minimization called dist...

Unlabeled Data Help: Minimax Analysis and Adversarial Robustness

The recent proposed self-supervised learning (SSL) approaches successful...

Adversarially Robust Estimate and Risk Analysis in Linear Regression

Adversarially robust learning aims to design algorithms that are robust ...

Please sign up or login with your details

Forgot password? Click here to reset