Generalized Beliefs for Cooperative AI

by   Darius Muglich, et al.

Self-play is a common paradigm for constructing solutions in Markov games that can yield optimal policies in collaborative settings. However, these policies often adopt highly-specialized conventions that make playing with a novel partner difficult. To address this, recent approaches rely on encoding symmetry and convention-awareness into policy training, but these require strong environmental assumptions and can complicate policy training. We therefore propose moving the learning of conventions to the belief space. Specifically, we propose a belief learning model that can maintain beliefs over rollouts of policies not seen at training time, and can thus decode and adapt to novel conventions at test time. We show how to leverage this model for both search and training of a best response over various pools of policies to greatly improve ad-hoc teamplay. We also show how our setup promotes explainability and interpretability of nuanced agent conventions.


page 15

page 16

page 17

page 18


On-the-fly Strategy Adaptation for ad-hoc Agent Coordination

Training agents in cooperative settings offers the promise of AI agents ...

Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents

Robustly cooperating with unseen agents and human partners presents sign...

Neural Recursive Belief States in Multi-Agent Reinforcement Learning

In multi-agent reinforcement learning, the problem of learning to act is...

Improving Policies via Search in Cooperative Partially Observable Games

Recent superhuman results in games have largely been achieved in a varie...

Decentralized Inference via Capability Type Structures in Cooperative Multi-Agent Systems

This work studies the problem of ad hoc teamwork in teams composed of ag...

Human-AI Coordination via Human-Regularized Search and Learning

We consider the problem of making AI agents that collaborate well with h...

Information Design in Crowdfunding under Thresholding Policies

In crowdfunding, an entrepreneur often has to decide how to disclose the...

Please sign up or login with your details

Forgot password? Click here to reset