Room Clearance with Feudal Hierarchical Reinforcement Learning

by   Henry Charlesworth, et al.

Reinforcement learning (RL) is a general framework that allows systems to learn autonomously through trial-and-error interaction with their environment. In recent years combining RL with expressive, high-capacity neural network models has led to impressive performance in a diverse range of domains. However, dealing with the large state and action spaces often required for problems in the real world still remains a significant challenge. In this paper we introduce a new simulation environment, "Gambit", designed as a tool to build scenarios that can drive RL research in a direction useful for military analysis. Using this environment we focus on an abstracted and simplified room clearance scenario, where a team of blue agents have to make their way through a building and ensure that all rooms are cleared of (and remain clear) of enemy red agents. We implement a multi-agent version of feudal hierarchical RL that introduces a command hierarchy where a commander at the higher level sends orders to multiple agents at the lower level who simply have to learn to follow these orders. We find that breaking the task down in this way allows us to solve a number of non-trivial floorplans that require the coordination of multiple agents much more efficiently than the standard baseline RL algorithms we compare with. We then go on to explore how qualitatively different behaviour can emerge depending on what we prioritise in the agent's reward function (e.g. clearing the building quickly vs. prioritising rescuing civilians).


page 1

page 3

page 5

page 6

page 7


Extended Markov Games to Learn Multiple Tasks in Multi-Agent Reinforcement Learning

The combination of Formal Methods with Reinforcement Learning (RL) has r...

Learning to Incentivize Other Learning Agents

The challenge of developing powerful and general Reinforcement Learning ...

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Joint Perception-Action Tasks

The purpose of this tutorial is to help individuals use the FireCommande...

Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systems

Reinforcement learning (RL) has been used in a range of simulated real-w...

Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning

In recent years we have seen fast progress on a number of benchmark prob...

Escape Room: A Configurable Testbed for Hierarchical Reinforcement Learning

Recent successes in Reinforcement Learning have encouraged a fast-growin...

Hearts Gym: Learning Reinforcement Learning as a Team Event

Amidst the COVID-19 pandemic, the authors of this paper organized a Rein...

Please sign up or login with your details

Forgot password? Click here to reset