FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

by   Zhenyu Li, et al.

Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93 We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. The code is open-sourced.


page 13

page 14

page 15


Few-shot In-context Learning for Knowledge Base Question Answering

Question answering over knowledge bases is considered a difficult proble...

BeamSearchQA: Large Language Models are Strong Zero-Shot QA Solver

Open-domain question answering is a crucial task that often requires acc...

Complementing GPT-3 with Few-Shot Sequence-to-Sequence Semantic Parsing over Wikidata

As the largest knowledge base, Wikidata is a massive source of knowledge...

ADMUS: A Progressive Question Answering Framework Adaptable to Multiple Knowledge Sources

With the introduction of deep learning models, semantic parsingbased kno...

Knowledge Base Question Answering for Space Debris Queries

Space agencies execute complex satellite operations that need to be supp...

Question Answering via Integer Programming over Semi-Structured Knowledge

Answering science questions posed in natural language is an important AI...

VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering

We present Visual Knowledge oriented Programming platform (VisKoP), a kn...

Please sign up or login with your details

Forgot password? Click here to reset