Quiz-Style Question Generation for News Stories

by   Adam D. Lelkes, et al.

A large majority of American adults get at least some of their news from the Internet. Even though many online news products have the goal of informing their users about the news, they lack scalable and reliable tools for measuring how well they are achieving this goal, and therefore have to resort to noisy proxy metrics (e.g., click-through rates or reading time) to track their performance. As a first step towards measuring news informedness at a scale, we study the problem of quiz-style multiple-choice question generation, which may be used to survey users about their knowledge of recent news. In particular, we formulate the problem as two sequence-to-sequence tasks: question-answer generation (QAG) and distractor, or incorrect answer, generation (DG). We introduce NewsQuizQA, the first dataset intended for quiz-style question-answer generation, containing 20K human written question-answer pairs from 5K news article summaries. Using this dataset, we propose a series of novel techniques for applying large pre-trained Transformer encoder-decoder models, namely PEGASUS and T5, to the tasks of question-answer generation and distractor generation. We show that our models outperform strong baselines using both automated metrics and human raters. We provide a case study of running weekly quizzes on real-world users via the Google Surveys platform over the course of two months. We found that users generally found the automatically generated questions to be educational and enjoyable. Finally, to serve the research community, we are releasing the NewsQuizQA dataset.


page 1

page 2

page 3

page 4


Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus

The ability to ask questions is important in both human and machine inte...

Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization

Generating educational questions of fairytales or storybooks is vital fo...

Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning

Motivated by suggested question generation in conversational news recomm...

A Feasibility Study of Answer-Agnostic Question Generation for Education

We conduct a feasibility study into the applicability of answer-agnostic...

NewsPod: Automatic and Interactive News Podcasts

News podcasts are a popular medium to stay informed and dive deep into n...

Generating News-Centric Crossword Puzzles As A Constraint Satisfaction and Optimization Problem

Crossword puzzles have traditionally served not only as entertainment bu...

Discord Questions: A Computational Approach To Diversity Analysis in News Coverage

There are many potential benefits to news readers accessing diverse sour...

Please sign up or login with your details

Forgot password? Click here to reset