Training Classifiers that are Universally Robust to All Label Noise Levels

by   Jingyi Xu, et al.

For classification tasks, deep neural networks are prone to overfitting in the presence of label noise. Although existing methods are able to alleviate this problem at low noise levels, they encounter significant performance reduction at high noise levels, or even at medium noise levels when the label noise is asymmetric. To train classifiers that are universally robust to all noise levels, and that are not sensitive to any variation in the noise model, we propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. In particular, we shall assume that a small subset of any given noisy dataset is known to have correct labels, which we treat as "positive", while the remaining noisy subset is treated as "unlabeled". Our framework consists of the following two components: (1) We shall generate, via iterative updates, an augmented clean subset with additional reliable "positive" samples filtered from "unlabeled" samples; (2) We shall train a teacher model on this larger augmented clean set. With the guidance of the teacher model, we then train a student model on the whole dataset. Experiments were conducted on the CIFAR-10 dataset with synthetic label noise at multiple noise levels for both symmetric and asymmetric noise. The results show that our framework generally outperforms at medium to high noise levels. We also evaluated our framework on Clothing1M, a real-world noisy dataset, and we achieved 2.94 state-of-the-art methods.


PENCIL: Deep Learning with Noisy Labels

Deep learning has achieved excellent performance in various computer vis...

ProMix: Combating Label Noise via Maximizing Clean Sample Utility

The ability to train deep neural networks under label noise is appealing...

Probabilistic End-to-end Noise Correction for Learning with Noisy Labels

Deep learning has achieved excellent performance in various computer vis...

Blind Knowledge Distillation for Robust Image Classification

Optimizing neural networks with noisy labels is a challenging task, espe...

TrustNet: Learning from Trusted Data Against (A)symmetric Label Noise

Robustness to label noise is a critical property for weakly-supervised c...

FPR Estimation for Fraud Detection in the Presence of Class-Conditional Label Noise

We consider the problem of estimating the false-/ true-positive-rate (FP...

A Topological Filter for Learning with Label Noise

Noisy labels can impair the performance of deep neural networks. To tack...

Please sign up or login with your details

Forgot password? Click here to reset