Uniform Convergence and Generalization for Nonconvex Stochastic Minimax Problems

05/28/2022
∙
by   Siqi Zhang, et al.
∙
0
∙

This paper studies the uniform convergence and generalization bounds for nonconvex-(strongly)-concave (NC-SC/NC-C) stochastic minimax optimization. We first establish the uniform convergence between the empirical minimax problem and the population minimax problem and show the 𝒊Ėƒ(dΚ^2Ïĩ^-2) and 𝒊Ėƒ(dÏĩ^-4) sample complexities respectively for the NC-SC and NC-C settings, where d is the dimension number and Κ is the condition number. To the best of our knowledge, this is the first uniform convergence measured by the first-order stationarity in stochastic minimax optimization. Based on the uniform convergence, we shed light on the sample and gradient complexities required for finding an approximate stationary point for stochastic minimax optimization in the NC-SC and NC-C settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
∙ 12/05/2022

A Simple and Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization

This paper studies the stochastic optimization for decentralized nonconv...
research
∙ 10/17/2022

Tight Analysis of Extra-gradient and Optimistic Gradient Methods For Nonconvex Minimax Problems

Despite the established convergence theory of Optimistic Gradient Descen...
research
∙ 03/29/2021

The Complexity of Nonconvex-Strongly-Concave Minimax Optimization

This paper studies the complexity for finding approximate stationary poi...
research
∙ 10/03/2022

On Stability and Generalization of Bilevel Optimization Problem

(Stochastic) bilevel optimization is a frequently encountered problem in...
research
∙ 04/21/2023

Near-Optimal Decentralized Momentum Method for Nonconvex-PL Minimax Problems

Minimax optimization plays an important role in many machine learning ta...
research
∙ 10/31/2022

Nesterov Meets Optimism: Rate-Optimal Optimistic-Gradient-Based Method for Stochastic Bilinearly-Coupled Minimax Optimization

We provide a novel first-order optimization algorithm for bilinearly-cou...
research
∙ 06/09/2022

What is a Good Metric to Study Generalization of Minimax Learners?

Minimax optimization has served as the backbone of many machine learning...

Please sign up or login with your details

Forgot password? Click here to reset