Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty

06/28/2019
by   Dan Hendrycks, et al.
5

Self-supervision provides effective representations for downstream tasks without requiring labels. However, existing approaches lag behind fully supervised training and are often not thought beneficial beyond obviating the need for annotations. We find that self-supervision can benefit robustness in a variety of ways, including robustness to adversarial examples, label corruption, and common input corruptions. Additionally, self-supervision greatly benefits out-of-distribution detection on difficult, near-distribution outliers, so much so that it exceeds the performance of fully supervised methods. These results demonstrate the promise of self-supervision for improving robustness and uncertainty estimation and establish these tasks as new axes of evaluation for future self-supervised learning research.

READ FULL TEXT
research
12/08/2021

On visual self-supervision and its effect on model robustness

Recent self-supervision methods have found success in learning feature r...
research
02/23/2023

Improving Adaptive Conformal Prediction Using Self-Supervised Learning

Conformal prediction is a powerful distribution-free tool for uncertaint...
research
03/31/2021

On the Origin of Species of Self-Supervised Learning

In the quiet backwaters of cs.CV, cs.LG and stat.ML, a cornucopia of new...
research
01/10/2020

Pruning Convolutional Neural Networks with Self-Supervision

Convolutional neural networks trained without supervision come close to ...
research
07/20/2022

Tailoring Self-Supervision for Supervised Learning

Recently, it is shown that deploying a proper self-supervision is a pros...
research
12/23/2022

Benchmark for Uncertainty Robustness in Self-Supervised Learning

Self-Supervised Learning (SSL) is crucial for real-world applications, e...
research
02/22/2019

Learning about an exponential amount of conditional distributions

We introduce the Neural Conditioner (NC), a self-supervised machine able...

Please sign up or login with your details

Forgot password? Click here to reset