Leveraging Demonstrations with Latent Space Priors

10/26/2022
by   Jonas Gehring, et al.
0

Demonstrations provide insight into relevant state or action space regions, bearing great potential to boost the efficiency and practicality of reinforcement learning agents. In this work, we propose to leverage demonstration datasets by combining skill learning and sequence modeling. Starting with a learned joint latent space, we separately train a generative model of demonstration sequences and an accompanying low-level policy. The sequence model forms a latent space prior over plausible demonstration behaviors to accelerate learning of high-level policies. We show how to acquire such priors from state-only motion capture demonstrations and explore several methods for integrating them into policy learning on transfer tasks. Our experimental results confirm that latent space priors provide significant gains in learning speed and final performance in a set of challenging sparse-reward environments with a complex, simulated humanoid. Videos, source code and pre-trained models are available at the corresponding project website at https://facebookresearch.github.io/latent-space-priors .

READ FULL TEXT

page 6

page 7

06/14/2023

Skill-Critic: Refining Learned Skills for Reinforcement Learning

Hierarchical reinforcement learning (RL) can accelerate long-horizon dec...
11/04/2022

Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics

Skill-based reinforcement learning (RL) has emerged as a promising strat...
06/15/2023

Behavioral Cloning via Search in Embedded Demonstration Dataset

Behavioural cloning uses a dataset of demonstrations to learn a behaviou...
10/05/2020

Mastering Atari with Discrete World Models

Intelligent agents need to generalize from past experience to achieve go...
11/02/2020

Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models

The potential benefits of model-free reinforcement learning to real robo...
10/18/2022

CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations

Although reinforcement learning has found widespread use in dense reward...
09/30/2019

Imagine That! Leveraging Emergent Affordances for Tool Synthesis in Reaching Tasks

In this paper we investigate an artificial agent's ability to perform ta...

Please sign up or login with your details

Forgot password? Click here to reset