Networked Multi-Agent Reinforcement Learning with Emergent Communication

by   Shubham Gupta, et al.

Multi-Agent Reinforcement Learning (MARL) methods find optimal policies for agents that operate in the presence of other learning agents. Central to achieving this is how the agents coordinate. One way to coordinate is by learning to communicate with each other. Can the agents develop a language while learning to perform a common task? In this paper, we formulate and study a MARL problem where cooperative agents are connected to each other via a fixed underlying network. These agents can communicate along the edges of this network by exchanging discrete symbols. However, the semantics of these symbols are not predefined and, during training, the agents are required to develop a language that helps them in accomplishing their goals. We propose a method for training these agents using emergent communication. We demonstrate the applicability of the proposed framework by applying it to the problem of managing traffic controllers, where we achieve state-of-the-art performance as compared to a number of strong baselines. More importantly, we perform a detailed analysis of the emergent communication to show, for instance, that the developed language is grounded and demonstrate its relationship with the underlying network topology. To the best of our knowledge, this is the only work that performs an in depth analysis of emergent communication in a networked MARL setting while being applicable to a broad class of problems.


page 1

page 2

page 3

page 4


Emergent Communication through Negotiation

Multi-agent reinforcement learning offers a way to study how communicati...

A Survey of Multi-Agent Reinforcement Learning with Communication

Communication is an effective mechanism for coordinating the behavior of...

Cooperative Multi-Agent Learning for Navigation via Structured State Abstraction

Cooperative multi-agent reinforcement learning (MARL) for navigation ena...

Establishing Shared Query Understanding in an Open Multi-Agent System

We propose a method that allows to develop shared understanding between ...

On Voting Strategies and Emergent Communication

Humans use language to collectively execute complex strategies in additi...

Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel

While multi-agent reinforcement learning has been used as an effective m...

An In-Depth Analysis of Discretization Methods for Communication Learning using Backpropagation with Multi-Agent Reinforcement Learning

Communication is crucial in multi-agent reinforcement learning when agen...

Please sign up or login with your details

Forgot password? Click here to reset