Adaptive Prior Selection for Repertoire-based Online Learning in Robotics

by   Rituraj Kaushik, et al.

Among the data-efficient approaches for online adaptation in robotics (meta-learning, model-based reinforcement learning, etc.), repertoire-based learning (1) generates a large and diverse set policies in simulation that acts as a "reservoir" for future adaptations and (2) learns to pick online the best working policies according to the current situation (e.g., a damaged robot, a new object, etc.). Each of these policies performs a different task, for instance, walking in different directions; these policies are then sequenced with a planning algorithm to achieve the given task. In this paper, we relax the assumption of previous works that a single repertoire is enough for adaptation. Instead, we generate repertoires for many different situations (e.g., with a missing leg, on different floors, etc.) in simulation that act as priors for adaptation. Our main contribution is an algorithm, APROL (Adaptive Prior selection for Repertoire-based Online Learning) to plan the next action by incorporating these priors when the robot has no information about the current situation. We evaluate APROL on two simulated tasks: (1) pushing unknown objects of various shapes and sizes with a kuka arm and (2) a goal reaching task with a damaged hexapod robot. We compare with "Reset-free Trial and Error" (RTE) and various single repertoire-based baselines. The results show that APROL solves both tasks in less interaction time than the baselines. Additionally, we demonstrate APROL on a real, damaged hexapod that quickly learns compensatory policies to reach a goal by avoiding obstacle in the path.


page 6

page 8


Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors

Meta-learning algorithms can accelerate the model-based reinforcement le...

SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulation

The framework of Simulation-to-real learning, i.e, learning policies in ...

Provably Efficient Model-based Policy Adaptation

The high sample complexity of reinforcement learning challenges its use ...

Learning to Adapt: Meta-Learning for Model-Based Control

Although reinforcement learning methods can achieve impressive results i...

Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments

This work developed a meta-learning approach that adapts the control pol...

Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search

One of the most interesting features of Bayesian optimization for direct...

From exploration to control: learning object manipulation skills through novelty search and local adaptation

Programming a robot to deal with open-ended tasks remains a challenge, i...

Please sign up or login with your details

Forgot password? Click here to reset