Decentralized Stochastic Variance Reduced Extragradient Method
share
This paper studies decentralized convex-concave minimax optimization problems of the form min_xmax_y f(x,y) ≜1/m∑_i=1^m f_i(x,y), where m is the number of agents and each local function can be written as f_i(x,y)=1/n∑_j=1^n f_i,j(x,y). We propose a novel decentralized optimization algorithm, called multi-consensus stochastic variance reduced extragradient, which achieves the best known stochastic first-order oracle (SFO) complexity for this problem. Specifically, each agent requires 𝒪((n+κ√(n))log(1/ε)) SFO calls for strongly-convex-strongly-concave problem and 𝒪((n+√(n)L/ε)log(1/ε)) SFO call for general convex-concave problem to achieve ε-accurate solution in expectation, where κ is the condition number and L is the smoothness parameter. The numerical experiments show the proposed method performs better than baselines.
READ FULL TEXT