Compositional Generalization and Decomposition in Neural Program Synthesis

by   Kensen Shi, et al.
berkeley college

When writing programs, people have the ability to tackle a new complex task by decomposing it into smaller and more familiar subtasks. While it is difficult to measure whether neural program synthesis methods have similar capabilities, what we can measure is whether they compositionally generalize, that is, whether a model that has been trained on the simpler subtasks is subsequently able to solve more complex tasks. In this paper, we focus on measuring the ability of learned program synthesizers to compositionally generalize. We first characterize several different axes along which program synthesis methods would be desired to generalize, e.g., length generalization, or the ability to combine known subroutines in new ways that do not occur in the training data. Based on this characterization, we introduce a benchmark suite of tasks to assess these abilities based on two popular existing datasets, SCAN and RobustFill. Finally, we make first attempts to improve the compositional generalization ability of Transformer models along these axes through novel attention mechanisms that draw inspiration from a human-like decomposition strategy. Empirically, we find our modified Transformer models generally perform better than natural baselines, but the tasks remain challenging.


page 15

page 16


ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis

When writing programs, people have the ability to tackle a new complex t...

Revisiting the Compositional Generalization Abilities of Neural Sequence Models

Compositional generalization is a fundamental trait in humans, allowing ...

Toward Compositional Generalization in Object-Oriented World Modeling

Compositional generalization is a critical ability in learning and decis...

Measuring Compositional Generalization: A Comprehensive Method on Realistic Data

State-of-the-art machine learning methods exhibit limited compositional ...

Transformer-based Program Synthesis for Low-Data Environments

Recent advancements in large pre-trained transformer models (GPT2/3, T5)...

ProTo: Program-Guided Transformer for Program-Guided Tasks

Programs, consisting of semantic and structural information, play an imp...

Please sign up or login with your details

Forgot password? Click here to reset