Leveraging Demonstrations with Latent Space Priors

by   Jonas Gehring, et al.

Demonstrations provide insight into relevant state or action space regions, bearing great potential to boost the efficiency and practicality of reinforcement learning agents. In this work, we propose to leverage demonstration datasets by combining skill learning and sequence modeling. Starting with a learned joint latent space, we separately train a generative model of demonstration sequences and an accompanying low-level policy. The sequence model forms a latent space prior over plausible demonstration behaviors to accelerate learning of high-level policies. We show how to acquire such priors from state-only motion capture demonstrations and explore several methods for integrating them into policy learning on transfer tasks. Our experimental results confirm that latent space priors provide significant gains in learning speed and final performance in a set of challenging sparse-reward environments with a complex, simulated humanoid. Videos, source code and pre-trained models are available at the corresponding project website at https://facebookresearch.github.io/latent-space-priors .


page 6

page 7


Skill-Critic: Refining Learned Skills for Reinforcement Learning

Hierarchical reinforcement learning (RL) can accelerate long-horizon dec...

Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics

Skill-based reinforcement learning (RL) has emerged as a promising strat...

Behavioral Cloning via Search in Embedded Demonstration Dataset

Behavioural cloning uses a dataset of demonstrations to learn a behaviou...

Mastering Atari with Discrete World Models

Intelligent agents need to generalize from past experience to achieve go...

Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models

The potential benefits of model-free reinforcement learning to real robo...

CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations

Although reinforcement learning has found widespread use in dense reward...

Imagine That! Leveraging Emergent Affordances for Tool Synthesis in Reaching Tasks

In this paper we investigate an artificial agent's ability to perform ta...

Please sign up or login with your details

Forgot password? Click here to reset