A Survey of Meta-Reinforcement Learning

by   Jacob Beck, et al.
University of Oxford
Stanford University

While deep reinforcement learning (RL) has fueled multiple high-profile successes in machine learning, it is held back from more widespread adoption by its often poor data efficiency and the limited generality of the policies it produces. A promising approach for alleviating these limitations is to cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible. In this survey, we describe the meta-RL problem setting in detail as well as its major variations. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. Using these clusters, we then survey meta-RL algorithms and applications. We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.


Robust Meta-Reinforcement Learning with Curriculum-Based Task Sampling

Meta-reinforcement learning (meta-RL) acquires meta-policies that show g...

Hindsight Foresight Relabeling for Meta-Reinforcement Learning

Meta-reinforcement learning (meta-RL) algorithms allow for agents to lea...

Learning to reinforcement learn

In recent years deep reinforcement learning (RL) systems have attained s...

Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning

Reinforcement Learning (RL) based solutions are being adopted in a varie...

Reinforcement Learning Algorithm Selection

This paper formalises the problem of online algorithm selection in the c...

RL^3: Boosting Meta Reinforcement Learning via RL inside RL^2

Meta reinforcement learning (meta-RL) methods such as RL^2 have emerged ...

Context Meta-Reinforcement Learning via Neuromodulation

Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt ...

Please sign up or login with your details

Forgot password? Click here to reset