Learning Feasibility to Imitate Demonstrators with Different Dynamics

by   Zhangjie Cao, et al.

The goal of learning from demonstrations is to learn a policy for an agent (imitator) by mimicking the behavior in the demonstrations. Prior works on learning from demonstrations assume that the demonstrations are collected by a demonstrator that has the same dynamics as the imitator. However, in many real-world applications, this assumption is limiting – to improve the problem of lack of data in robotics, we would like to be able to leverage demonstrations collected from agents with different dynamics. This can be challenging as the demonstrations might not even be feasible for the imitator. Our insight is that we can learn a feasibility metric that captures the likelihood of a demonstration being feasible by the imitator. We develop a feasibility MDP (f-MDP) and derive the feasibility score by learning an optimal policy in the f-MDP. Our proposed feasibility measure encourages the imitator to learn from more informative demonstrations, and disregard the far from feasible demonstrations. Our experiments on four simulated environments and on a real robot show that the policy learned with our approach achieves a higher expected return than prior works. We show the videos of the real robot arm experiments on our website (https://sites.google.com/view/learning-feasibility).


page 2

page 4

page 6

page 7

page 14


Learning from Imperfect Demonstrations from Agents with Varying Dynamics

Imitation learning enables robots to learn from demonstrations. Previous...

"Guess what I'm doing": Extending legibility to sequential decision tasks

In this paper we investigate the notion of legibility in sequential deci...

Learning from Imperfect Demonstrations via Adversarial Confidence Transfer

Existing learning from demonstration algorithms usually assume access to...

Out-of-Dynamics Imitation Learning from Multimodal Demonstrations

Existing imitation learning works mainly assume that the demonstrator wh...

Imitation Learning with a Value-Based Prior

The goal of imitation learning is for an apprentice to learn how to beha...

Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations

Multimodal demonstrations provide robots with an abundance of informatio...

One Policy to Dress Them All: Learning to Dress People with Diverse Poses and Garments

Robot-assisted dressing could benefit the lives of many people such as o...

Please sign up or login with your details

Forgot password? Click here to reset