ASPiRe:Adaptive Skill Priors for Reinforcement Learning

by   Mengda Xu, et al.

We introduce ASPiRe (Adaptive Skill Prior for RL), a new approach that leverages prior experience to accelerate reinforcement learning. Unlike existing methods that learn a single skill prior from a large and diverse dataset, our framework learns a library of different distinction skill priors (i.e., behavior priors) from a collection of specialized datasets, and learns how to combine them to solve a new task. This formulation allows the algorithm to acquire a set of specialized skill priors that are more reusable for downstream tasks; however, it also brings up additional challenges of how to effectively combine these unstructured sets of skill priors to form a new prior for new tasks. Specifically, it requires the agent not only to identify which skill prior(s) to use but also how to combine them (either sequentially or concurrently) to form a new prior. To achieve this goal, ASPiRe includes Adaptive Weight Module (AWM) that learns to infer an adaptive weight assignment between different skill priors and uses them to guide policy learning for downstream tasks via weighted Kullback-Leibler divergences. Our experiments demonstrate that ASPiRe can significantly accelerate the learning of new downstream tasks in the presence of multiple priors and show improvement on competitive baselines.


page 3

page 8

page 9

page 18


Accelerating Reinforcement Learning with Learned Skill Priors

Intelligent agents rely heavily on prior experience when learning a new ...

Learning to Navigate Using Mid-Level Visual Priors

How much does having visual priors about the world (e.g. the fact that t...

Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery

Current reinforcement learning (RL) in robotics often experiences diffic...

Stochastic Neural Networks for Hierarchical Reinforcement Learning

Deep reinforcement learning has achieved many impressive results in rece...

Hierarchical Few-Shot Imitation with Skill Transition Models

A desirable property of autonomous agents is the ability to both solve l...

Reinforcement Learning from Passive Data via Latent Intentions

Passive observational data, such as human videos, is abundant and rich i...

SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Exploration

The ability to effectively reuse prior knowledge is a key requirement wh...

Please sign up or login with your details

Forgot password? Click here to reset