Compositional Semantic Parsing with Large Language Models

by   Andrew Drozdov, et al.

Humans can reason compositionally when presented with new tasks. Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artificial compositional generalization tasks such as SCAN. In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and refine these prompting techniques to address them. Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse. This method allows us to set a new state of the art for CFQ while requiring only 1 traditional approaches. Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.


page 1

page 2

page 3

page 4


Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures

While mainstream machine learning methods are known to have limited abil...

On graph-based reentrancy-free semantic parsing

We propose a novel graph-based approach for semantic parsing that resolv...

Training Naturalized Semantic Parsers with Very Little Data

Semantic parsing is an important NLP problem, particularly for voice ass...

On the Compositional Generalization Gap of In-Context Learning

Pretrained large generative language models have shown great performance...

Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing

Despite their strong performance on many tasks, pre-trained language mod...

Learning to Generalize Compositionally by Transferring Across Semantic Parsing Tasks

Neural network models often generalize poorly to mismatched domains or d...

Please sign up or login with your details

Forgot password? Click here to reset