Implicit Communication as Minimum Entropy Coupling

by   Samuel Sokota, et al.
University of Oxford
Carnegie Mellon University

In many common-payoff games, achieving good performance requires players to develop protocols for communicating their private information implicitly – i.e., using actions that have non-communicative effects on the environment. Multi-agent reinforcement learning practitioners typically approach this problem using independent learning methods in the hope that agents will learn implicit communication as a byproduct of expected return maximization. Unfortunately, independent learning methods are incapable of doing this in many settings. In this work, we isolate the implicit communication problem by identifying a class of partially observable common-payoff games, which we call implicit referential games, whose difficulty can be attributed to implicit communication. Next, we introduce a principled method based on minimum entropy coupling that leverages the structure of implicit referential games, yielding a new perspective on implicit communication. Lastly, we show that this method can discover performant implicit communication protocols in settings with very large spaces of messages.


Agent Environment Cycle Games

Partially Observable Stochastic Games (POSGs), are the most general mode...

Learning Efficient Multi-agent Communication: An Information Bottleneck Approach

Many real-world multi-agent reinforcement learning applications require ...

Low Entropy Communication in Multi-Agent Reinforcement Learning

Communication in multi-agent reinforcement learning has been drawing att...

FCMNet: Full Communication Memory Net for Team-Level Cooperation in Multi-Agent Systems

Decentralized cooperation in partially-observable multi-agent systems re...

Inductive Bias for Emergent Communication in a Continuous Setting

We study emergent communication in a multi-agent reinforcement learning ...

Haptic communication optimises joint decisions and affords implicit confidence sharing

Group decisions can outperform the choices of the best individual group ...

Please sign up or login with your details

Forgot password? Click here to reset