research
∙
08/17/2023
Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models
We analyze the dynamics of streaming stochastic gradient descent (SGD) i...
share
research
∙
06/20/2022
Only Tails Matter: Average-Case Universality and Robustness in the Convex Regime
The recently developed average-case analysis of optimization methods all...
research
∙
06/15/2022
Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions
Stochastic gradient descent (SGD) is a pillar of modern machine learning...
research
∙
06/02/2022
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
We analyze the dynamics of large batch stochastic gradient descent with ...
research
∙
05/14/2022
Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties
We develop a stochastic differential equation, called homogenized SGD, f...
research
∙
06/07/2021
Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models
We analyze a class of stochastic gradient algorithms with momentum on a ...
research
∙
02/08/2021
SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality
We propose a new framework, inspired by random matrix theory, for analyz...
research
∙
06/08/2020
Halting Time is Predictable for Large Models: A Universality Property and Average-case Analysis
Average-case analysis computes the complexity of an algorithm averaged o...
research
∙
03/23/2020
A termination criterion for stochastic gradient descent for binary classification
We propose a new, simple, and computationally inexpensive termination te...
research
∙
03/31/2017