Explicit Syntactic Guidance for Neural Text Generation

by   Yafu Li, et al.

Most existing text generation models follow the sequence-to-sequence paradigm. Generative Grammar suggests that humans generate natural language texts by learning language grammar. We propose a syntax-guided generation schema, which generates the sequence guided by a constituency parse tree in a top-down direction. The decoding process can be decomposed into two parts: (1) predicting the infilling texts for each constituent in the lexicalized syntax context given the source sentence; (2) mapping and expanding each constituent to construct the next-level syntax context. Accordingly, we propose a structural beam search method to find possible syntax structures hierarchically. Experiments on paraphrase generation and machine translation show that the proposed method outperforms autoregressive baselines, while also demonstrating effectiveness in terms of interpretability, controllability, and diversity.


Syntax-guided Controlled Generation of Paraphrases

Given a sentence (e.g., "I like mangoes") and a constraint (e.g., sentim...

Transformer-Based Neural Text Generation with Syntactic Guidance

We study the problem of using (partial) constituency parse trees as synt...

Syntax-driven Iterative Expansion Language Models for Controllable Text Generation

The dominant language modeling paradigms handle text as a sequence of di...

Syntax Controlled Knowledge Graph-to-Text Generation with Order and Semantic Consistency

The knowledge graph (KG) stores a large amount of structural knowledge, ...

TreeGAN: Syntax-Aware Sequence Generation with Generative Adversarial Networks

Generative Adversarial Networks (GANs) have shown great capacity on imag...

Progressive Transformer-Based Generation of Radiology Reports

Inspired by Curriculum Learning, we propose a consecutive (i.e. image-to...

Simulated annealing for optimization of graphs and sequences

Optimization of discrete structures aims at generating a new structure w...

Please sign up or login with your details

Forgot password? Click here to reset