Adapting to game trees in zero-sum imperfect information games

12/23/2022
by   Côme Fiegel, et al.
0

Imperfect information games (IIG) are games in which each player only partially observes the current game state. We study how to learn ϵ-optimal strategies in a zero-sum IIG through self-play with trajectory feedback. We give a problem-independent lower bound 𝒪(H(A_𝒳+B_𝒴)/ϵ^2) on the required number of realizations to learn these strategies with high probability, where H is the length of the game, A_𝒳 and B_𝒴 are the total number of actions for the two players. We also propose two Follow the Regularize leader (FTRL) algorithms for this setting: Balanced-FTRL which matches this lower bound, but requires the knowledge of the information set structure beforehand to define the regularization; and Adaptive-FTRL which needs 𝒪(H^2(A_𝒳+B_𝒴)/ϵ^2) plays without this requirement by progressively adapting the regularization to the observations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/19/2020

From Poincaré Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization

In this paper we investigate the Follow the Regularized Leader dynamics ...
research
09/01/2023

Local and adaptive mirror descents in extensive-form games

We study how to learn ϵ-optimal strategies in zero-sum imperfect informa...
research
10/21/2020

Solving Zero-Sum One-Sided Partially Observable Stochastic Games

Many security and other real-world situations are dynamic in nature and ...
research
06/11/2021

Model-Free Learning for Two-Player Zero-Sum Partially Observable Markov Games with Perfect Recall

We study the problem of learning a Nash equilibrium (NE) in an imperfect...
research
03/19/2023

Instance-dependent Sample Complexity Bounds for Zero-sum Matrix Games

We study the sample complexity of identifying an approximate equilibrium...
research
01/18/2018

On a Generic Security Game Model

To protect the systems exposed to the Internet against attacks, a securi...
research
12/18/2021

Playing Against Fair Adversaries in Stochastic Games with Total Rewards

We investigate zero-sum turn-based two-player stochastic games in which ...

Please sign up or login with your details

Forgot password? Click here to reset