Internet-augmented language models through few-shot prompting for open-domain question answering

by   Angeliki Lazaridou, et al.

In this work, we aim to capitalize on the unique few-shot capabilities offered by large-scale language models to overcome some of their challenges with respect to grounding to factual and up-to-date information. Motivated by semi-parametric language models, which ground their decisions in external retrieved evidence, we use few-shot prompting to learn to condition language models on information returned from the web using Google Search, a broad and constantly updated knowledge source. Our approach does not involve fine-tuning or learning additional parameters, thus making it applicable to any language model, offering like this a strong baseline. Indeed, we find that language models conditioned on the web surpass performance of closed-book models of similar, or even larger, model sizes in open-domain question answering. Finally, we find that increasing the inference-time compute of models, achieved via using multiple retrieved evidences to generate multiple answers followed by a reranking stage, alleviates generally decreased performance of smaller few-shot language models. All in all, our findings suggest that it might be beneficial to slow down the race towards the biggest model and instead shift the attention towards finding more effective ways to use models, including but not limited to better prompting or increasing inference-time compute.


page 1

page 2

page 3

page 4


Evaluation of medium-large Language Models at zero-shot closed book generative question answering

Large language models (LLMs) have garnered significant attention, but th...

Better Question-Answering Models on a Budget

Low-rank adaptation (LoRA) and question-answer datasets from large langu...

Knowledge Sanitization of Large Language Models

We explore a knowledge sanitization approach to mitigate the privacy con...

Language Models as Fact Checkers?

Recent work has suggested that language models (LMs) store both common-s...

Ask Me Anything: A simple strategy for prompting language models

Large language models (LLMs) transfer well to new tasks out-of-the-box s...

GPT4GEO: How a Language Model Sees the World's Geography

Large language models (LLMs) have shown remarkable capabilities across a...

Holistic Evaluation of Language Models

Language models (LMs) are becoming the foundation for almost all major l...

Please sign up or login with your details

Forgot password? Click here to reset