Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data

by   David Lowell, et al.

Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions on (a) observed (unlabeled) examples; and (b) corresponding 'noised' examples produced via data augmentation. While UDA has gained popularity for text classification, open questions linger over which design decisions are necessary and over how to extend the method to sequence labeling tasks. This method has recently gained traction for text classification. In this paper, we re-examine UDA and demonstrate its efficacy on several sequential tasks. Our main contribution is an empirical study of UDA to establish which components of the algorithm confer benefits in NLP. Notably, although prior work has emphasized the use of clever augmentation techniques including back-translation, we find that enforcing consistency between predictions assigned to observed and randomly substituted words often yields comparable (or greater) benefits compared to these complex perturbation models. Furthermore, we find that applying its consistency loss affords meaningful gains without any unlabeled data at all, i.e., in a standard supervised setting. In short: UDA need not be unsupervised, and does not require complex data augmentation to be effective.


page 1

page 2

page 3

page 4


Unsupervised Data Augmentation

Despite its success, deep learning still needs large labeled datasets to...

MixPUL: Consistency-based Augmentation for Positive and Unlabeled Learning

Learning from positive and unlabeled data (PU learning) is prevalent in ...

Semi-Supervised Models via Data Augmentationfor Classifying Interactive Affective Responses

We present semi-supervised models with data augmentation (SMDA), a semi-...

Deep Active Learning with Augmentation-based Consistency Estimation

In active learning, the focus is mainly on the selection strategy of unl...

Consistent Text Categorization using Data Augmentation in e-Commerce

The categorization of massive e-Commerce data is a crucial, well-studied...

SanitAIs: Unsupervised Data Augmentation to Sanitize Trojaned Neural Networks

The application of self-supervised methods has resulted in broad improve...

Please sign up or login with your details

Forgot password? Click here to reset