A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks

by   Tushar Semwal, et al.

Transfer Learning (TL) plays a crucial role when a given dataset has insufficient labeled examples to train an accurate model. In such scenarios, the knowledge accumulated within a model pre-trained on a source dataset can be transferred to a target dataset, resulting in the improvement of the target model. Though TL is found to be successful in the realm of image-based applications, its impact and practical use in Natural Language Processing (NLP) applications is still a subject of research. Due to their hierarchical architecture, Deep Neural Networks (DNN) provide flexibility and customization in adjusting their parameters and depth of layers, thereby forming an apt area for exploiting the use of TL. In this paper, we report the results and conclusions obtained from extensive empirical experiments using a Convolutional Neural Network (CNN) and try to uncover thumb rules to ensure a successful positive transfer. In addition, we also highlight the flawed means that could lead to a negative transfer. We explore the transferability of various layers and describe the effect of varying hyper-parameters on the transfer performance. Also, we present a comparison of accuracy value and model size against state-of-the-art methods. Finally, we derive inferences from the empirical results and provide best practices to achieve a successful positive transfer.


page 1

page 2

page 3

page 4


Transfer learning for time series classification

Transfer learning for deep neural networks is the process of first train...

How Transferable are Neural Networks in NLP Applications?

Transfer learning is aimed to make use of valuable knowledge in a source...

Medical Text Classification using Convolutional Neural Networks

We present an approach to automatically classify clinical text at a sent...

Improving Transferability of Deep Neural Networks

Learning from small amounts of labeled data is a challenge in the area o...

Overfreezing Meets Overparameterization: A Double Descent Perspective on Transfer Learning of Deep Neural Networks

We study the generalization behavior of transfer learning of deep neural...

A Novel Deep Learning Method for Textual Sentiment Analysis

Sentiment analysis is known as one of the most crucial tasks in the fiel...

Interpretation of Swedish Sign Language using Convolutional Neural Networks and Transfer Learning

The automatic interpretation of sign languages is a challenging task, as...

Please sign up or login with your details

Forgot password? Click here to reset