Lazy-CFR: a fast regret minimization algorithm for extensive games with imperfect information

by   Yichi Zhou, et al.

In this paper, we focus on solving two-player zero-sum extensive games with imperfect information. Counterfactual regret minimization (CFR) is the most popular algorithm on solving such games and achieves state-of-the-art performance in practice. However, the performance of CFR is not fully understood, since empirical results on the regret are much better than the upper bound proved in zinkevich2008regret. Another issue of CFR is that CFR has to traverse the whole game tree in each round, which is not tolerable in large scale games. In this paper, we present a novel technique, lazy update, which can avoid traversing the whole game tree in CFR. Further, we present a novel analysis on the CFR with lazy update. Our analysis can also be applied to the vanilla CFR, which results in a much tighter regret bound than that proved in zinkevich2008regret. Inspired by lazy update, we further present a novel CFR variant, named Lazy-CFR. Compared to traversing O(|I|) information sets in vanilla CFR, Lazy-CFR needs only to traverse O(√(|I|)) information sets per round while the regret bound almost keep the same, where I is the class of all information sets. As a result, Lazy-CFR shows better convergence result compared with vanilla CFR. Experimental results consistently show that Lazy-CFR outperforms the vanilla CFR significantly.


page 1

page 2

page 3

page 4


No-Regret Learning in Extensive-Form Games with Imperfect Recall

Counterfactual Regret Minimization (CFR) is an efficient no-regret learn...

Solving Imperfect-Information Games via Discounted Regret Minimization

Counterfactual regret minimization (CFR) is a family of iterative algori...

Fast Algorithms for Poker Require Modelling it as a Sequential Bayesian Game

Many recent results in imperfect information games were only formulated ...

Revisiting CFR+ and Alternating Updates

The CFR+ algorithm for solving imperfect information games is a variant ...

Alternative Function Approximation Parameterizations for Solving Games: An Analysis of f-Regression Counterfactual Regret Minimization

Function approximation is a powerful approach for structuring large deci...

Combining No-regret and Q-learning

Counterfactual Regret Minimization (CFR) has found success in settings l...

Conic Blackwell Algorithm: Parameter-Free Convex-Concave Saddle-Point Solving

We develop new parameter and scale-free algorithms for solving convex-co...

Please sign up or login with your details

Forgot password? Click here to reset