Learning from Demonstration with Weakly Supervised Disentanglement

by   Yordan Hristov, et al.

Robotic manipulation tasks, such as wiping with a soft sponge, require control from multiple rich sensory modalities. Human-robot interaction, aimed at teaching robots, is difficult in this setting as there is potential for mismatch between human and machine comprehension of the rich data streams. We treat the task of interpretable learning from demonstration as an optimisation problem over a probabilistic generative model. To account for the high-dimensionality of the data, a high-capacity neural network is chosen to represent the model. The latent variables in this model are explicitly aligned with high-level notions and concepts that are manifested in a set of demonstrations. We show that such alignment is best achieved through the use of labels from the end user, in an appropriately restricted vocabulary, in contrast to the conventional approach of the designer picking a prior over the latent variables. Our approach is evaluated in the context of a table-top robot manipulation task performed by a PR2 robot – that of dabbing liquids with a sponge (forcefully pressing a sponge and moving it along a surface). The robot provides visual information, arm joint positions and arm joint efforts. We have made videos of the task and data available - see supplementary materials at https://sites.google.com/view/weak-label-lfd


page 6

page 15

page 16


Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

In recent years, we have seen an emergence of data-driven approaches in ...

HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration

The ability to learn from human demonstration endows robots with the abi...

Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube

We build a system that enables any human to control a robot hand and arm...

TACO: Learning Task Decomposition via Temporal Alignment for Control

Many advanced Learning from Demonstration (LfD) methods consider the dec...

Interpretable Latent Spaces for Learning from Demonstration

Effective human-robot interaction, such as in robot learning from human ...

Learning Robot Geometry as Distance Fields: Applications to Whole-body Manipulation

In this work, we propose to learn robot geometry as distance fields (RDF...

Heterogeneous Learning from Demonstration

The development of human-robot systems able to leverage the strengths of...

Please sign up or login with your details

Forgot password? Click here to reset