research
∙
02/07/2023
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy
We present an algorithm for minimizing an objective with hard-to-compute...
share
research
∙
06/15/2022
Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays
The existing analysis of asynchronous stochastic gradient descent (SGD) ...
research
∙
04/11/2022
Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares
We consider potentially non-convex optimization problems, for which opti...
research
∙
10/07/2021
A Stochastic Newton Algorithm for Distributed Convex Optimization
We propose and analyze a stochastic Newton algorithm for homogeneous dis...
research
∙
09/01/2021
The Minimax Complexity of Distributed Optimization
In this thesis, I study the minimax oracle complexity of distributed sto...
research
∙
06/04/2021
An Even More Optimal Stochastic Optimization Algorithm: Minibatching and Interpolation Learning
We present and analyze an algorithm for optimizing smooth and convex or ...
research
∙
02/19/2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent
Recent work has highlighted the role of initialization scale in determin...
research
∙
02/02/2021
The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication
We resolve the min-max complexity of distributed stochastic convex optim...
research
∙
07/13/2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
We provide a detailed asymptotic study of gradient flow trajectories and...
research
∙
06/08/2020
Minibatch vs Local SGD for Heterogeneous Distributed Learning
We analyze Local SGD (aka parallel or federated SGD) and Minibatch SGD i...
research
∙
04/02/2020
Mirrorless Mirror Descent: A More Natural Discretization of Riemannian Gradient Flow
We present a direct (primal only) derivation of Mirror Descent as a "par...
research
∙
02/20/2020
Kernel and Rich Regimes in Overparametrized Models
A recent line of work studies overparametrized neural networks in the "k...
research
∙
02/18/2020
Is Local SGD Better than Minibatch SGD?
We study local SGD (also known as parallel SGD and federated averaging),...
research
∙
12/05/2019
Lower Bounds for Non-Convex Stochastic Optimization
We lower bound the complexity of finding ϵ-stationary points (with gradi...
research
∙
11/06/2019
The gradient complexity of linear regression
We investigate the computational complexity of several basic linear alge...
research
∙
07/01/2019
Open Problem: The Oracle Complexity of Convex Optimization with Limited Memory
We note that known methods achieving the optimal oracle complexity for f...
research
∙
06/21/2019
Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis
We design a general framework for answering adaptive statistical queries...
research
∙
06/13/2019
Kernel and Deep Regimes in Overparametrized Models
A recent line of work studies overparametrized neural networks in the "k...
research
∙
02/13/2019
The Complexity of Making the Gradient Small in Stochastic Convex Optimization
We give nearly matching upper and lower bounds on the oracle complexity ...
research
∙
06/29/2018
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
Classifiers can be trained with data-dependent constraints to satisfy fa...
research
∙
05/25/2018
Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization
We suggest a general oracle-based framework that captures different para...
research
∙
03/12/2018
The Everlasting Database: Statistical Validity at a Fair Price
The problem of handling adaptivity in data analysis, intentional or not,...
research
∙
05/25/2017
Implicit Regularization in Matrix Factorization
We study implicit regularization when optimizing an underdetermined quad...
research
∙
05/25/2016