DeepAI AI Chat
Log In Sign Up

Hypothesis Only Baselines in Natural Language Inference

by   Adam Poliak, et al.
Johns Hopkins University

We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.


page 1

page 2

page 3

page 4


Exploring Lexical Irregularities in Hypothesis-Only Models of Natural Language Inference

Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE)...

Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference

Natural Language Inference (NLI) datasets often contain hypothesis-only ...

A Puzzle-Based Dataset for Natural Language Inference

We provide here a dataset for tasks related to natural language understa...

Smoothing Entailment Graphs with Language Models

The diversity and Zipfian frequency distribution of natural language pre...

Misleading Failures of Partial-input Baselines

Recent work establishes dataset difficulty and removes annotation artifa...

SemEval-2023 Task 7: Multi-Evidence Natural Language Inference for Clinical Trial Data

This paper describes the results of SemEval 2023 task 7 – Multi-Evidence...