Specializing Versatile Skill Libraries using Local Mixture of Experts

by   Onur Celik, et al.

A long-cherished vision in robotics is to equip robots with skills that match the versatility and precision of humans. For example, when playing table tennis, a robot should be capable of returning the ball in various ways while precisely placing it at the desired location. A common approach to model such versatile behavior is to use a Mixture of Experts (MoE) model, where each expert is a contextual motion primitive. However, learning such MoEs is challenging as most objectives force the model to cover the entire context space, which prevents specialization of the primitives resulting in rather low-quality components. Starting from maximum entropy reinforcement learning (RL), we decompose the objective into optimizing an individual lower bound per mixture component. Further, we introduce a curriculum by allowing the components to focus on a local context region, enabling the model to learn highly accurate skill representations. To this end, we use local context distributions that are adapted jointly with the expert primitives. Our lower bound advocates an iterative addition of new components, where new components will concentrate on local context regions not covered by the current MoE. This local and incremental learning results in a modular MoE model of high accuracy and versatility, where both properties can be scaled by adding more components on the fly. We demonstrate this by an extensive ablation and on two challenging simulated robot skill learning tasks. We compare our achieved performance to LaDiPS and HiREPS, a known hierarchical policy search method for learning diverse skills.


page 7

page 8

page 18


Curriculum-Based Imitation of Versatile Skills

Learning skills by imitation is a promising concept for the intuitive te...

Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning

We present a reinforcement learning (RL) framework that enables quadrupe...

Information Maximizing Curriculum: A Curriculum-Based Approach for Training Mixtures of Experts

Mixtures of Experts (MoE) are known for their ability to learn complex c...

Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks

In this paper, we study the problem of learning a repertoire of low-leve...

Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors

Humans intuitively solve tasks in versatile ways, varying their behavior...

CoMic: Complementary Task Learning & Mimicry for Reusable Skills

Learning to control complex bodies and reuse learned behaviors is a long...

A practical DMPs Implementation for Skill Creation and Teleoperation with Assistive Manipulators

Assistive robotic manipulators are becoming increasingly important for p...

Please sign up or login with your details

Forgot password? Click here to reset