Extractive Summarization via ChatGPT for Faithful Summary Generation

by   Haopeng Zhang, et al.

Extractive summarization is a crucial task in natural language processing that aims to condense long documents into shorter versions by directly extracting sentences. The recent introduction of ChatGPT has attracted significant interest in the NLP community due to its remarkable performance on a wide range of downstream tasks. However, concerns regarding factuality and faithfulness have hindered its practical applications for summarization systems. This paper first presents a thorough evaluation of ChatGPT's performance on extractive summarization and compares it with traditional fine-tuning methods on various benchmark datasets. Our experimental analysis reveals that ChatGPT's extractive summarization performance is still inferior to existing supervised systems in terms of ROUGE scores. In addition, we explore the effectiveness of in-context learning and chain-of-thought reasoning for enhancing its performance. Furthermore, we find that applying an extract-then-generate pipeline with ChatGPT yields significant performance improvements over abstractive baselines in terms of summary faithfulness. These observations highlight potential directions for enhancing ChatGPT's capabilities for faithful text summarization tasks using two-stage approaches.


page 1

page 2

page 3

page 4


Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization

Text summarization has been a crucial problem in natural language proces...

Text Summarization using Abstract Meaning Representation

With an ever increasing size of text present on the Internet, automatic ...

Hybrid MemNet for Extractive Summarization

Extractive text summarization has been an extensive research problem in ...

Transformer Based Implementation for Automatic Book Summarization

Document Summarization is the procedure of generating a meaningful and c...

Reinforced Abstractive Summarization with Adaptive Length Controlling

Document summarization, as a fundamental task in natural language genera...

Applying Transformer-based Text Summarization for Keyphrase Generation

Keyphrases are crucial for searching and systematizing scholarly documen...

Enhancing Generation through Summarization Duality and Explicit Outline Control

Automatically open-ended long text generation poses significant challeng...

Please sign up or login with your details

Forgot password? Click here to reset