Calibration of Shared Equilibria in General Sum Partially Observable Markov Games

by   Nelson Vadori, et al.

Training multi-agent systems (MAS) to achieve realistic equilibria gives us a useful tool to understand and model real-world systems. We consider a general sum partially observable Markov game where agents of different types share a single policy network, conditioned on agent-specific information. This paper aims at i) formally understanding equilibria reached by such agents, and ii) matching emergent phenomena of such equilibria to real-world targets. Parameter sharing with decentralized execution has been introduced as an efficient way to train multiple agents using a single policy network. However, the nature of resulting equilibria reached by such agents has not been yet studied: we introduce the novel concept of Shared equilibrium as a symmetric pure Nash equilibrium of a certain Functional Form Game (FFG) and prove convergence to the latter for a certain class of games using self-play. In addition, it is important that such equilibria satisfy certain constraints so that MAS are calibrated to real world data for practical use: we solve this problem by introducing a novel dual-Reinforcement Learning based approach that fits emergent behaviors of agents in a Shared equilibrium to externally-specified targets, and apply our methods to a n-player market example. We do so by calibrating parameters governing distributions of agent types rather than individual agents, which allows both behavior differentiation among agents and coherent scaling of the shared policy network to multiple agents.


page 7

page 8

page 17

page 18

page 19


Emergent Prosociality in Multi-Agent Games Through Gifting

Coordination is often critical to forming prosocial behaviors – behavior...

Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria

Multiagent learning settings are inherently more difficult than single-a...

Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning

Real economies can be modeled as a sequential imperfect-information game...

End-to-End Learning and Intervention in Games

In a social system, the self-interest of agents can be detrimental to th...

Minimax Theorem for Latent Games or: How I Learned to Stop Worrying about Mixed-Nash and Love Neural Nets

Adversarial training, a special case of multi-objective optimization, is...

Learning Density-Based Correlated Equilibria for Markov Games

Correlated Equilibrium (CE) is a well-established solution concept that ...

Infer Your Enemies and Know Yourself, Learning in Real-Time Bidding with Partially Observable Opponents

Real-time bidding, as one of the most popular mechanisms for selling onl...

Please sign up or login with your details

Forgot password? Click here to reset