Jonathan Berant

research

∙ 06/23/2023

Long-range Language Modeling with Self-retrieval

Retrieval-augmented language models (LMs) have received much attention r...

0 Ohad Rubin, et al. ∙

research

∙ 05/31/2023

From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces

Much of the previous work towards digital agents for graphical user inte...

0 Peter Shaw, et al. ∙

research

∙ 05/23/2023

ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding

We introduce ZeroSCROLLS, a zero-shot benchmark for natural language und...

0 Uri Shaham, et al. ∙

research

∙ 04/25/2023

Answering Questions by Meta-Reasoning over Multiple Chains of Thought

Modern systems for multi-hop question answering (QA) typically break que...

5 Ori Yoran, et al. ∙

research

∙ 01/30/2023

Crawling the Internal Knowledge-Base of Language Models

Language models are trained on large volumes of text, and as a result th...

0 Roi Cohen, et al. ∙

research

∙ 12/20/2022

What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary

Dual encoders are now the dominant architecture for dense retrieval. Yet...

0 Ori Ram, et al. ∙

research

∙ 12/13/2022

Diverse Demonstrations Improve In-context Compositional Generalization

In-context learning has shown great success in i.i.d semantic parsing sp...

0 Itay Levy, et al. ∙

research

∙ 12/01/2022

Simplifying and Understanding State Space Models with Diagonal Linear RNNs

Sequence models based on linear state spaces (SSMs) have recently emerge...

0 Ankit Gupta, et al. ∙

research

∙ 11/01/2022

Training Vision-Language Models with Less Bimodal Supervision

Standard practice in pretraining multimodal models, such as vision-langu...

0 Elad Segal, et al. ∙

research

∙ 09/06/2022

Analyzing Transformers in Embedding Space

Understanding Transformer-based models has attracted significant attenti...

10 Guy Dar, et al. ∙

research

∙ 08/01/2022

Efficient Long-Text Understanding with Short-Text Models

Transformer-based pretrained language models (LMs) are ubiquitous across...

0 Maor Ivgi, et al. ∙

research

∙ 05/25/2022

QAMPARI: : An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs

Existing benchmarks for open-domain question answering (ODQA) typically ...

4 Samuel Joseph Amouyal, et al. ∙

research

∙ 04/28/2022

Inferring Implicit Relations with Language Models

A prominent challenge for modern language understanding systems is the a...

7 Uri Katz, et al. ∙

research

∙ 02/13/2022

Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments

Neural scaling laws define a predictable relationship between a model's ...

0 Maor Ivgi, et al. ∙

research

∙ 01/15/2022

Unobserved Local Structures Make Compositional Generalization Hard

While recent work has convincingly showed that sequence-to-sequence mode...

3 Ben Bogin, et al. ∙

research

∙ 01/10/2022

SCROLLS: Standardized CompaRison Over Long Language Sequences

NLP benchmarks have largely focused on short texts, such as sentences an...

0 Uri Shaham, et al. ∙

research

∙ 12/16/2021

Learning To Retrieve Prompts for In-Context Learning

In-context learning is a recent paradigm in natural language understandi...

0 Ohad Rubin, et al. ∙

research

∙ 12/14/2021

Learning to Retrieve Passages without Supervision

Dense retrievers for open-domain question answering (ODQA) have been sho...

1 Ori Ram, et al. ∙

research

∙ 12/12/2021

Weakly Supervised Mapping of Natural Language to SQL through Question Decomposition

Natural Language Interfaces to Databases (NLIDBs), where users pose quer...

0 Tomer Wolfson, et al. ∙

research

∙ 09/22/2021

COVR: A test-bed for Visually Grounded Compositional Generalization with real images

While interest in models that generalize at test time to new composition...

1 Ben Bogin, et al. ∙

research

∙ 09/06/2021

Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization

Modern semantic parsers suffer from two principal limitations. First, tr...

0 Inbar Oren, et al. ∙

research

∙ 07/29/2021

Break, Perturb, Build: Automatic Perturbation of Reasoning Paths through Question Decomposition

Recent efforts to create challenge benchmarks that test the abilities of...

0 Mor Geva, et al. ∙

research

∙ 07/15/2021

Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Models pre-trained with a language modeling objective possess ample worl...

0 Ori Yoran, et al. ∙

research

∙ 06/13/2021

Memory-efficient Transformers via Top-k Attention

Following the success of dot-product attention in Transformers, numerous...

0 Ankit Gupta, et al. ∙

research

∙ 04/17/2021

Question Decomposition with Dependency Graphs

QDMR is a meaning representation for complex questions, which decomposes...

0 Matan Hasson, et al. ∙

research

∙ 04/13/2021

What's in your Head? Emergent Behaviour in Multi-Task Transformer Models

The primary paradigm for multi-task training in natural language process...

0 Mor Geva, et al. ∙

research

∙ 04/11/2021

Achieving Model Robustness through Discrete Adversarial Training

Discrete adversarial attacks are symbolic perturbations to a language in...

0 Maor Ivgi, et al. ∙

research

∙ 03/17/2021

Value-aware Approximate Attention

Following the success of dot-product attention in Transformers, numerous...

0 Ankit Gupta, et al. ∙

research

∙ 03/09/2021

BERTese: Learning to Speak to BERT

Large pre-trained language models have been shown to encode large amount...

0 Adi Haviv, et al. ∙

research

∙ 01/06/2021

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies

A key limitation in current datasets for multi-hop reasoning is that the...

0 Mor Geva, et al. ∙

research

∙ 01/02/2021

Few-Shot Question Answering by Pretraining Span Selection

In a number of question answering (QA) benchmarks, pretrained models hav...

0 Ori Ram, et al. ∙

research

∙ 12/29/2020

Transformer Feed-Forward Layers Are Key-Value Memories

Feed-forward layers constitute two-thirds of a transformer model's param...

29 Mor Geva, et al. ∙

research

∙ 10/23/2020

SmBoP: Semi-autoregressive Bottom-up Semantic Parsing

The de-facto standard decoding method for semantic parsing in recent yea...

0 Ohad Rubin, et al. ∙

research

∙ 10/12/2020

Improving Compositional Generalization in Semantic Parsing

Generalization of models to out-of-distribution (OOD) data has captured ...

0 Inbar Oren, et al. ∙

research

∙ 09/30/2020

Learning Object Detection from Captions via Textual Scene Attributes

Object detection is a fundamental task in computer vision, requiring lar...

9 Achiya Jerbi, et al. ∙

research

∙ 09/23/2020

Scene Graph to Image Generation with Contextualized Object Layout Refinement

Generating high-quality images from scene graphs, that is, graphs that d...

23 Maor Ivgi, et al. ∙

research

∙ 09/13/2020

Span-based Semantic Parsing for Compositional Generalization

Despite the success of sequence-to-sequence (seq2seq) models in semantic...

0 Jonathan Herzig, et al. ∙

research

∙ 09/02/2020

A Simple Global Neural Discourse Parser

Discourse parsing is largely dominated by greedy parsers with manually-d...

0 Yichu Zhou, et al. ∙

research

∙ 07/01/2020

Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering

Answering questions that involve multi-step reasoning requires decomposi...

7 Ben Bogin, et al. ∙

research

∙ 06/11/2020

Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge

To what extent can a neural network systematically reason over symbolic ...

0 Alon Talmor, et al. ∙

research

∙ 06/05/2020

GMAT: Global Memory Augmentation for Transformers

Transformer-based models have become ubiquitous in natural language proc...

8 Jonathan Berant, et al. ∙

research

∙ 05/02/2020

Obtaining Faithful Interpretations from Compositional Neural Networks

Neural module networks (NMNs) are a popular approach for modeling compos...

22 Sanjay Subramanian, et al. ∙

research

∙ 04/12/2020

Explaining Question Answering Models through Text Generation

Large pre-trained language models (LMs) have been shown to perform surpr...

24 Veronica Latcinnik, et al. ∙

research

∙ 04/09/2020

Injecting Numerical Reasoning Skills into Language Models

Large pre-trained language models (LMs) are known to encode substantial ...

0 Mor Geva, et al. ∙

research

∙ 04/06/2020

Evaluating the Evaluation of Diversity in Natural Language Generation

Despite growing interest in natural language generation (NLG) models tha...

0 Guy Tevet, et al. ∙

research

∙ 04/06/2020

Evaluating NLP Models via Contrast Sets

Standard test sets for supervised learning evaluate in-distribution gene...

0 Matt Gardner, et al. ∙

research

∙ 01/31/2020

Break It Down: A Question Understanding Benchmark

Understanding natural language questions entails the ability to break do...

0 Tomer Wolfson, et al. ∙

research

∙ 12/31/2019

oLMpics – On what Language Model Pre-training Captures

Recent success of pre-trained language models (LMs) has spurred widespre...

14 Alon Talmor, et al. ∙

research

∙ 10/05/2019

On the Limits of Learning to Actively Learn Semantic Representations

One of the goals of natural language understanding is to develop models ...

0 Omri Koshorek, et al. ∙

research

∙ 09/25/2019

Question Answering is a Format; When is it Useful?

Recent years have seen a dramatic expansion of tasks and datasets posed ...

0 Matt Gardner, et al. ∙

Jonathan Berant

Featured Co-authors