DeepAI AI Chat
Log In Sign Up

Answering Questions by Meta-Reasoning over Multiple Chains of Thought

by   Ori Yoran, et al.

Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer. Often, multiple chains are sampled and aggregated through a voting mechanism over the final answers, but the intermediate steps themselves are discarded. While such approaches improve performance, they do not consider the relations between intermediate steps across chains and do not provide a unified explanation for the predicted answer. We introduce Multi-Chain Reasoning (MCR), an approach which prompts large language models to meta-reason over multiple chains of thought, rather than aggregating their answers. MCR examines different reasoning chains, mixes information between them and selects the most relevant facts in generating an explanation and predicting the answer. MCR outperforms strong baselines on 7 multi-hop QA datasets. Moreover, our analysis reveals that MCR explanations exhibit high quality, enabling humans to verify its answers.


page 1

page 2

page 3

page 4


Do Multi-hop Readers Dream of Reasoning Chains?

General Question Answering (QA) systems over texts require the multi-hop...

Multi-hop Question Answering via Reasoning Chains

Multi-hop question answering requires models to gather information from ...

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

When answering a question, humans utilize the information available acro...

Complexity-Based Prompting for Multi-Step Reasoning

We study the task of prompting large-scale language models to perform mu...

Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering

Despite the rapid progress in multihop question-answering (QA), models s...

Best of Both Worlds: A Hybrid Approach for Multi-Hop Explanation with Declarative Facts

Language-enabled AI systems can answer complex, multi-hop questions to h...

Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning

Our goal is a question-answering (QA) system that can show how its answe...