Learn to Code-Switch: Data Augmentation using Copy Mechanism on Language Modeling

10/24/2018
by   Genta Indra Winata, et al.
0

Building large-scale datasets for training code-switching language models is challenging and very expensive. To alleviate this problem parallel corpus has been a major workaround. However, existing solutions use linguistic constraints which may not capture the real data distribution. In this work, we propose a novel method for learning how to generate code-switching sentences from parallel corpora. Our model uses a Seq2Seq in combination with pointer networks to align and choose words from the monolingual sentences and form a grammatical code-switching sentence. In our experiment, we show that by training a language model using the generated sentences improve the perplexity score by around 10

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset

Sign in with Google

×

Use your Google Account to sign in to DeepAI

×

Consider DeepAI Pro