REIN-2: Giving Birth to Prepared Reinforcement Learning Agents Using Reinforcement Learning Agents

10/11/2021
by   Aristotelis Lazaridis, et al.
0

Deep Reinforcement Learning (Deep RL) has been in the spotlight for the past few years, due to its remarkable abilities to solve problems which were considered to be practically unsolvable using traditional Machine Learning methods. However, even state-of-the-art Deep RL algorithms have various weaknesses that prevent them from being used extensively within industry applications, with one such major weakness being their sample-inefficiency. In an effort to patch these issues, we integrated a meta-learning technique in order to shift the objective of learning to solve a task into the objective of learning how to learn to solve a task (or a set of tasks), which we empirically show that improves overall stability and performance of Deep RL algorithms. Our model, named REIN-2, is a meta-learning scheme formulated within the RL framework, the goal of which is to develop a meta-RL agent (meta-learner) that learns how to produce other RL agents (inner-learners) that are capable of solving given environments. For this task, we convert the typical interaction of an RL agent with the environment into a new, single environment for the meta-learner to interact with. Compared to traditional state-of-the-art Deep RL algorithms, experimental results show remarkable performance of our model in popular OpenAI Gym environments in terms of scoring and sample efficiency, including the Mountain Car hard-exploration environment.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2018

Some Considerations on Learning to Explore via Meta-Reinforcement Learning

We consider the problem of exploration in meta reinforcement learning. T...
research
06/05/2020

Rapid Task-Solving in Novel Environments

When thrust into an unfamiliar environment and charged with solving a se...
research
04/08/2021

Efficient time stepping for numerical integration using reinforcement learning

Many problems in science and engineering require the efficient numerical...
research
10/09/2019

Improving Generalization in Meta Reinforcement Learning using Learned Objectives

Biological evolution has distilled the experiences of many learners into...
research
01/29/2023

Sample Efficient Deep Reinforcement Learning via Local Planning

The focus of this work is sample-efficient deep reinforcement learning (...
research
05/03/2019

Meta-learners' learning dynamics are unlike learners'

Meta-learning is a tool that allows us to build sample-efficient learnin...
research
02/04/2021

Alchemy: A structured task distribution for meta-reinforcement learning

There has been rapidly growing interest in meta-learning as a method for...

Please sign up or login with your details

Forgot password? Click here to reset