TextNAS: A Neural Architecture Search Space tailored for Text Representation

by   Yujing Wang, et al.
ETH Zurich
Peking University

Learning text representation is crucial for text classification and other language related tasks. There are a diverse set of text representation networks in the literature, and how to find the optimal one is a non-trivial problem. Recently, the emerging Neural Architecture Search (NAS) techniques have demonstrated good potential to solve the problem. Nevertheless, most of the existing works of NAS focus on the search algorithms and pay little attention to the search space. In this paper, we argue that the search space is also an important human prior to the success of NAS in different applications. Thus, we propose a novel search space tailored for text representation. Through automatic search, the discovered network architecture outperforms state-of-the-art models on various public datasets on text classification and natural language inference tasks. Furthermore, some of the design principles found in the automatic network agree well with human intuition.


page 1

page 2

page 3

page 4


Automatic Subspace Evoking for Efficient Neural Architecture Search

Neural Architecture Search (NAS) aims to automatically find effective ar...

DDNAS: Discretized Differentiable Neural Architecture Search for Text Classification

Neural Architecture Search (NAS) has shown promising capability in learn...

Evolutionary-Neural Hybrid Agents for Architecture Search

Neural Architecture Search has recently shown potential to automate the ...

Architecture representations for quantum convolutional neural networks

The Quantum Convolutional Neural Network (QCNN) is a quantum circuit mod...

Neural Architecture Refinement: A Practical Way for Avoiding Overfitting in NAS

Neural architecture search (NAS) is proposed to automate the architectur...

A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

Deep learning has made major breakthroughs and progress in many fields. ...

AutoCaption: Image Captioning with Neural Architecture Search

Image captioning transforms complex visual information into abstract nat...

Please sign up or login with your details

Forgot password? Click here to reset