SummIt: Iterative Text Summarization via ChatGPT

by   Haopeng Zhang, et al.

Existing text summarization systems have made significant progress in recent years but typically generates summaries in a single step. The one-shot summarization setting is sometimes inadequate, however, as the generated summary may contain hallucinations or overlook important details related to the reader's interests. In this paper, we address this limitation by proposing SummIt, an iterative text summarization framework based on large language models like ChatGPT. Our framework enables the model to refine the generated summary iteratively through self-evaluation and feedback, closely resembling the iterative process humans undertake when drafting and revising summaries. We also explore using in-context learning to guide the rationale generation and summary refinement. Furthermore, we explore the potential benefits of integrating knowledge and topic extractors into the framework to enhance summary faithfulness and controllability. We evaluate the performance of our framework on three benchmark summarization datasets through empirical and qualitative analyses. We also conduct a human evaluation to validate the effectiveness of the model's refinements and find a potential issue of over-correction. Our code is available at <>.


page 1

page 2

page 3

page 4


EmailSum: Abstractive Email Thread Summarization

Recent years have brought about an interest in the challenging task of s...

Self-Refine: Iterative Refinement with Self-Feedback

Like people, LLMs do not always generate the best text for a given gener...

Summary Explorer: Visualizing the State of the Art in Text Summarization

This paper introduces Summary Explorer, a new tool to support the manual...

Interactive Editing for Text Summarization

Summarizing lengthy documents is a common and essential task in our dail...

Video Summarization via Actionness Ranking

To automatically produce a brief yet expressive summary of a long video,...

Topic-Aware Evaluation and Transformer Methods for Topic-Controllable Summarization

Topic-controllable summarization is an emerging research area with a wid...

Please sign up or login with your details

Forgot password? Click here to reset