Memory Asymmetry Creates Heteroclinic Orbits to Nash Equilibrium in Learning in Zero-Sum Games

by   Yuma Fujimoto, et al.

Learning in games considers how multiple agents maximize their own rewards through repeated games. Memory, an ability that an agent changes his/her action depending on the history of actions in previous games, is often introduced into learning to explore more clever strategies and discuss the decision-making of real agents like humans. However, such games with memory are hard to analyze because they exhibit complex phenomena like chaotic dynamics or divergence from Nash equilibrium. In particular, how asymmetry in memory capacities between agents affects learning in games is still unclear. In response, this study formulates a gradient ascent algorithm in games with asymmetry memory capacities. To obtain theoretical insights into learning dynamics, we first consider a simple case of zero-sum games. We observe complex behavior, where learning dynamics draw a heteroclinic connection from unstable fixed points to stable ones. Despite this complexity, we analyze learning dynamics and prove local convergence to these stable fixed points, i.e., the Nash equilibria. We identify the mechanism driving this convergence: an agent with a longer memory learns to exploit the other, which in turn endows the other's utility function with strict concavity. We further numerically observe such convergence in various initial strategies, action numbers, and memory lengths. This study reveals a novel phenomenon due to memory asymmetry, providing fundamental strides in learning in games and new insights into computing equilibria.


page 1

page 2

page 3

page 4


Learning in Multi-Memory Games Triggers Complex Dynamics Diverging from Nash Equilibrium

Repeated games consider a situation where multiple agents are motivated ...

Unpredictable dynamics in congestion games: memory loss can prevent chaos

We study the dynamics of simple congestion games with two resources wher...

Learning Nash Equilibria in Monotone Games

We consider multi-agent decision making where each agent's cost function...

The Nash Equilibrium with Inertia in Population Games

In the traditional game-theoretic set up, where agents select actions an...

Imitation dynamics in population games on community networks

We study the asymptotic behavior of deterministic, continuous-time imita...

Path to Stochastic Stability: Comparative Analysis of Stochastic Learning Dynamics in Games

Stochastic stability is a popular solution concept for stochastic learni...

Classifying Convergence Complexity of Nash Equilibria in Graphical Games Using Distributed Computing Theory

Graphical games are a useful framework for modeling the interactions of ...

Please sign up or login with your details

Forgot password? Click here to reset