Perseus: A Simple High-Order Regularization Method for Variational Inequalities

by   Tianyi Lin, et al.

This paper settles an open and challenging question pertaining to the design of simple high-order regularization methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding x^⋆∈𝒳 such that ⟨ F(x), x - x^⋆⟩≥ 0 for all x ∈𝒳 and we consider the setting where F: ℝ^d ↦ℝ^d is smooth with up to (p-1)^th-order derivatives. For the case of p = 2, <cit.> extended the cubic regularized Newton's method to VIs with a global rate of O(ϵ^-1). <cit.> proposed another second-order method which achieved an improved rate of O(ϵ^-2/3log(1/ϵ)), but this method required a nontrivial binary search procedure as an inner loop. High-order methods based on similar binary search procedures have been further developed and shown to achieve a rate of O(ϵ^-2/(p+1)log(1/ϵ)). However, such search procedure can be computationally prohibitive in practice and the problem of finding a simple high-order regularization methods remains as an open and challenging question in optimization theory. We propose a p^th-order method which does not require any binary search scheme and is guaranteed to converge to a weak solution with a global rate of O(ϵ^-2/(p+1)). A version with restarting attains a global linear and local superlinear convergence rate for smooth and strongly monotone VIs. Further, our method achieves a global rate of O(ϵ^-2/p) for solving smooth and non-monotone VIs satisfying the Minty condition; moreover, the restarted version again attains a global linear and local superlinear convergence rate if the strong Minty condition holds.


page 1

page 2

page 3

page 4


A Continuous-Time Perspective on Optimal Methods for Monotone Equation Problems

We study rescaled gradient dynamical systems in a Hilbert space ℋ, where...

Optimal Methods for Higher-Order Smooth Monotone Variational Inequalities

In this work, we present new simple and optimal algorithms for solving t...

On the convergence of single-call stochastic extra-gradient methods

Variational inequalities have recently attracted considerable interest i...

Revisiting the ACVI Method for Constrained Variational Inequalities

ACVI is a recently proposed first-order method for solving variational i...

Order of uniform approximation by polynomial interpolation in the complex plane and beyond

For Lagrange polynomial interpolation on open arcs X=γ in , it is well-k...

On Monotone Inclusions, Acceleration and Closed-Loop Control

We propose and analyze a new dynamical system with a closed-loop control...

Please sign up or login with your details

Forgot password? Click here to reset