Attention Option-Critic

by   Raviteja Chunduru, et al.
McGill University

Temporal abstraction in reinforcement learning is the ability of an agent to learn and use high-level behaviors, called options. The option-critic architecture provides a gradient-based end-to-end learning method to construct options. We propose an attention-based extension to this framework, which enables the agent to learn to focus different options on different aspects of the observation space. We show that this leads to behaviorally diverse options which are also capable of state abstraction, and prevents the degeneracy problems of option domination and frequent option switching that occur in option-critic, while achieving a similar sample complexity. We also demonstrate the more efficient, interpretable, and reusable nature of the learned options in comparison with option-critic, through different transfer learning tasks. Experimental results in a relatively simple four-rooms environment and the more complex ALE (Arcade Learning Environment) showcase the efficacy of our approach.


page 4

page 5

page 7

page 9

page 11

page 12

page 13


Diversity-Enriched Option-Critic

Temporal abstraction allows reinforcement learning agents to represent k...

Context-Specific Representation Abstraction for Deep Option Learning

Hierarchical reinforcement learning has focused on discovering temporall...

Learning to Explore by Reinforcement over High-Level Options

Autonomous 3D environment exploration is a fundamental task for various ...

Delegated Online Search

In a delegation problem, a principal P with commitment power tries to pi...

Option-Driven Design: Context, Tradeoffs, and Considerations for Accessibility

In Option-Driven Design, users must interact with options and settings f...

Safe Option-Critic: Learning Safety in the Option-Critic Architecture

Designing hierarchical reinforcement learning algorithms that induce a n...

PODNet: A Neural Network for Discovery of Plannable Options

Learning from demonstration has been widely studied in machine learning ...

Please sign up or login with your details

Forgot password? Click here to reset