Conservative Optimistic Policy Optimization via Multiple Importance Sampling

03/04/2021
βˆ™
by   Achraf Azize, et al.
βˆ™
0
βˆ™

Reinforcement Learning (RL) has been able to solve hard problems such as playing Atari games or solving the game of Go, with a unified approach. Yet modern deep RL approaches are still not widely used in real-world applications. One reason could be the lack of guarantees on the performance of the intermediate executed policies, compared to an existing (already working) baseline policy. In this paper, we propose an online model-free algorithm that solves conservative exploration in the policy optimization problem. We show that the regret of the proposed approach is bounded by π’ͺΜƒ(√(T)) for both discrete and continuous parameter spaces.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
βˆ™ 02/08/2020

Conservative Exploration in Reinforcement Learning

While learning in an unknown Markov Decision Process (MDP), an agent sho...
research
βˆ™ 06/24/2019

Deep Conservative Policy Iteration

Conservative Policy Iteration (CPI) is a founding algorithm of Approxima...
research
βˆ™ 05/22/2023

Policy Representation via Diffusion Probability Model for Reinforcement Learning

Popular reinforcement learning (RL) algorithms tend to produce a unimoda...
research
βˆ™ 12/15/2020

Policy Optimization as Online Learning with Mediator Feedback

Policy Optimization (PO) is a widely used approach to address continuous...
research
βˆ™ 08/27/2020

The Advantage Regret-Matching Actor-Critic

Regret minimization has played a key role in online learning, equilibriu...
research
βˆ™ 11/11/2020

Hamiltonian Q-Learning: Leveraging Importance-sampling for Data Efficient RL

Model-free reinforcement learning (RL), in particular Q-learning is wide...
research
βˆ™ 09/20/2022

A Joint Imitation-Reinforcement Learning Framework for Reduced Baseline Regret

In various control task domains, existing controllers provide a baseline...

Please sign up or login with your details

Forgot password? Click here to reset