Enhancing Generation through Summarization Duality and Explicit Outline Control

by   Yunzhe Li, et al.
University of Illinois at Urbana-Champaign
Alibaba Group
The Regents of the University of California

Automatically open-ended long text generation poses significant challenges due to semantic incoherence and plot implausibility. Previous works usually alleviate this problem through outlines in the form of short phrases or abstractive signals by designing unsupervised tasks, which tend to be unstable and weakly interpretable. Assuming that a summary serves as a mature outline, we introduce a two-stage, summary-enhanced outline supervised generation framework. This framework leverages the dual characteristics of the summarization task to improve outline prediction, resulting in more explicit and plausible outlines. Furthermore, we identify an underutilization issue in outline-based generation with both standard pretrained language models (e.g., GPT-2, BART) and large language models (e.g., Vicuna, ChatGPT). To address this, we propose a novel explicit outline control method for more effective utilization of generated outlines.


page 1

page 2

page 3

page 4


Hierarchical Text Generation using an Outline

Many challenges in natural language processing require generating text, ...

Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy

To mitigate potential risks associated with language models, recent AI d...

Factuality Enhanced Language Models for Open-Ended Text Generation

Pretrained language models (LMs) are susceptible to generate text with n...

DOC: Improving Long Story Coherence With Detailed Outline Control

We propose the Detailed Outline Control (DOC) framework for improving lo...

Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models

Large language models produce human-like text that drive a growing numbe...

Unsupervised Melody-to-Lyric Generation

Automatic melody-to-lyric generation is a task in which song lyrics are ...

Extractive Summarization via ChatGPT for Faithful Summary Generation

Extractive summarization is a crucial task in natural language processin...

Please sign up or login with your details

Forgot password? Click here to reset