UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation

by   Daixuan Cheng, et al.
University of the Chinese Academy of Sciences

Large Language Models (LLMs) are popular for their impressive abilities, but the need for model-specific fine-tuning or task-specific prompt engineering can hinder their generalization. We propose UPRISE (Universal Prompt Retrieval for Improving zero-Shot Evaluation), which tunes a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. Specifically, we demonstrate universality in a cross-task and cross-model scenario: the retriever is tuned on a diverse set of tasks, but tested on unseen task types; we use a small frozen LLM, GPT-Neo-2.7B, for tuning the retriever, but test the retriever on different LLMs of much larger scales, such as BLOOM-7.1B, OPT-66B and GPT3-175B. Additionally, we show that UPRISE mitigates the hallucination problem in our experiments with ChatGPT, suggesting its potential to improve even the strongest LLMs. Our model and code are available at https://github.com/microsoft/LMOps.


page 1

page 2

page 3

page 4


Retrieval of Soft Prompt Enhances Zero-Shot Task Generalization

During zero-shot inference with language models (LMs), using hard prompt...

Modular Retrieval for Generalization and Interpretation

New retrieval tasks have always been emerging, thus urging the developme...

RSpell: Retrieval-augmented Framework for Domain Adaptive Chinese Spelling Check

Chinese Spelling Check (CSC) refers to the detection and correction of s...

Taken by Surprise: Contrast effect for Similarity Scores

Accurately evaluating the similarity of object vector embeddings is of c...

Zero-shot NLG evaluation through Pairware Comparisons with LLMs

Evaluating Natural Language Generation (NLG) outputs is crucial but labo...

Improving Zero-Shot Generalization for CLIP with Synthesized Prompts

With the growing interest in pretrained vision-language models like CLIP...

DePT: Decoupled Prompt Tuning

This work breaks through the Base-New Tradeoff (BNT)dilemma in prompt tu...

Please sign up or login with your details

Forgot password? Click here to reset