Flexible Option Learning

by   Martin Klissarov, et al.
McGill University

Temporal abstraction in reinforcement learning (RL), offers the promise of improving generalization and knowledge transfer in complex environments, by propagating information more efficiently over time. Although option learning was initially formulated in a way that allows updating many options simultaneously, using off-policy, intra-option learning (Sutton, Precup Singh, 1999), many of the recent hierarchical reinforcement learning approaches only update a single option at a time: the option currently executing. We revisit and extend intra-option learning in the context of deep reinforcement learning, in order to enable updating all options consistent with current primitive action choices, without introducing any additional estimates. Our method can therefore be naturally adopted in most hierarchical RL frameworks. When we combine our approach with the option-critic algorithm for option discovery, we obtain significant improvements in performance and data-efficiency across a wide variety of domains.


page 9

page 16

page 17

page 18


Disentangling Options with Hellinger Distance Regularizer

In reinforcement learning (RL), temporal abstraction still remains as an...

Classifying Options for Deep Reinforcement Learning

In this paper we combine one method for hierarchical reinforcement learn...

Interpretable Option Discovery using Deep Q-Learning and Variational Autoencoders

Deep Reinforcement Learning (RL) is unquestionably a robust framework to...

The Eigenoption-Critic Framework

Eigenoptions (EOs) have been recently introduced as a promising idea for...

Hierarchical Reinforcement Learning via Advantage-Weighted Information Maximization

Real-world tasks are often highly structured. Hierarchical reinforcement...

Intrinsically motivated option learning: a comparative study of recent methods

Options represent a framework for reasoning across multiple time scales ...

Multitask Soft Option Learning

We present Multitask Soft Option Learning (MSOL), a hierarchical multita...

Please sign up or login with your details

Forgot password? Click here to reset