Explicit Second-Order Min-Max Optimization Methods with Optimal Convergence Guarantee

by   Tianyi Lin, et al.

We propose and analyze exact and inexact regularized Newton-type methods for finding a global saddle point of a convex-concave unconstrained min-max optimization problem. Compared to their first-order counterparts, investigations of second-order methods for min-max optimization are relatively limited, as obtaining global rates of convergence with second-order information is much more involved. In this paper, we highlight how second-order information can be used to speed up the dynamics of dual extrapolation methods despite inexactness. Specifically, we show that the proposed algorithms generate iterates that remain within a bounded set and the averaged iterates converge to an ϵ-saddle point within O(ϵ^-2/3) iterations in terms of a gap function. Our algorithms match the theoretically established lower bound in this context and our analysis provides a simple and intuitive convergence analysis for second-order methods without requiring any compactness assumptions. Finally, we present a series of numerical experiments on synthetic and real data that demonstrate the efficiency of the proposed algorithms.


page 1

page 2

page 3

page 4


The limits of min-max optimization algorithms: convergence to spurious non-critical sets

Compared to minimization problems, the min-max landscape in machine lear...

Potential Function-based Framework for Making the Gradients Small in Convex and Min-Max Optimization

Making the gradients small is a fundamental optimization problem that ha...

Higher-order methods for convex-concave min-max optimization and monotone variational inequalities

We provide improved convergence rates for constrained convex-concave min...

Optimality and Stability in Non-Convex-Non-Concave Min-Max Optimization

Convergence to a saddle point for convex-concave functions has been stud...

A Second-order Equilibrium in Nonconvex-Nonconcave Min-max Optimization: Existence and Algorithm

Min-max optimization, with a nonconvex-nonconcave objective function f: ...

Min-Max Optimization Made Simple: Approximating the Proximal Point Method via Contraction Maps

In this paper we present a first-order method that admits near-optimal c...

Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions

Stochastic Bilevel optimization usually involves minimizing an upper-lev...

Please sign up or login with your details

Forgot password? Click here to reset