Faster Stochastic Algorithms for Minimax Optimization under Polyak–Łojasiewicz Conditions
share
This paper considers stochastic first-order algorithms for minimax optimization under Polyak–Łojasiewicz (PL) conditions. We propose SPIDER-GDA for solving the finite-sum problem of the form min_x max_y f(x,y)≜1/n∑_i=1^n f_i(x,y), where the objective function f(x,y) is μ_x-PL in x and μ_y-PL in y; and each f_i(x,y) is L-smooth. We prove SPIDER-GDA could find an ϵ-optimal solution within 𝒪((n + √(n) κ_xκ_y^2)log (1/ϵ)) stochastic first-order oracle (SFO) complexity, which is better than the state-of-the-art method whose SFO upper bound is 𝒪((n + n^2/3κ_xκ_y^2)log (1/ϵ)), where κ_x≜ L/μ_x and κ_y≜ L/μ_y. For the ill-conditioned case, we provide an accelerated algorithm to reduce the computational cost further. It achieves 𝒪̃((n+√(n) κ_xκ_y)log^2 (1/ϵ)) SFO upper bound when κ_y ≳√(n). Our ideas also can be applied to the more general setting that the objective function only satisfies PL condition for one variable. Numerical experiments validate the superiority of proposed methods.
READ FULL TEXT