Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function

by   Devendra Singh Sachan, et al.

In this paper, we study bidirectional LSTM network for the task of text classification using both supervised and semi-supervised approaches. Several prior works have suggested that either complex pretraining schemes using unsupervised methods such as language modeling (Dai and Le 2015; Miyato, Dai, and Goodfellow 2016) or complicated models (Johnson and Zhang 2017) are necessary to achieve a high classification accuracy. However, we develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results compared with more complex approaches. Furthermore, in addition to cross-entropy loss, by using a combination of entropy minimization, adversarial, and virtual adversarial losses for both labeled and unlabeled data, we report state-of-the-art results for text classification task on several benchmark datasets. In particular, on the ACL-IMDB sentiment analysis and AG-News topic classification datasets, our method outperforms current approaches by a substantial margin. We also show the generality of the mixed objective function by improving the performance on relation extraction task.


page 1

page 2

page 3

page 4


KDSTM: Neural Semi-supervised Topic Modeling with Knowledge Distillation

In text classification tasks, fine tuning pretrained language models lik...

Variational Autoencoders for Semi-supervised Text Classification

Although semi-supervised variational autoencoder (SemiVAE) works in imag...

Stop Overcomplicating Selective Classification: Use Max-Logit

We tackle the problem of Selective Classification where the goal is to a...

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport

Recently, Neural Topic Models (NTM), inspired by variational autoencoder...

Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning

We propose a general approach to modeling semi-supervised learning (SSL)...

Unsupervised Representation Adversarial Learning Network: from Reconstruction to Generation

A good representation for arbitrarily complicated data should have the c...

NoPropaganda at SemEval-2020 Task 11: A Borrowed Approach to Sequence Tagging and Text Classification

This paper describes our contribution to SemEval-2020 Task 11: Detection...

Please sign up or login with your details

Forgot password? Click here to reset