research
∙
08/06/2023
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
In this work, we investigate the dynamics of stochastic gradient descent...
share
research
∙
06/16/2023
Trained Transformers Learn Linear Models In-Context
Attention-based neural networks such as transformers have demonstrated a...
research
∙
03/02/2023
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization
Linear classifiers and leaky ReLU networks trained by gradient flow on t...
research
∙
03/02/2023
The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks
In this work, we study the implications of the implicit bias of gradient...
research
∙
02/15/2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks
In this work, we provide a characterization of the feature-learning proc...
research
∙
02/11/2022
Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data
Benign overfitting, the phenomenon where interpolating models generalize...
research
∙
06/25/2021
Self-training Converts Weak Learners to Strong Learners in Mixture Models
We consider a binary classification problem when the data comes from a m...
research
∙
06/25/2021
Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent
Although the optimization objectives for learning neural networks are hi...
research
∙
04/19/2021
Provable Robustness of Adversarial Training for Learning Halfspaces with Noise
We analyze the properties of adversarial training for learning adversari...
research
∙
01/04/2021
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise
We consider a one-hidden-layer leaky ReLU network of arbitrary width tra...
research
∙
10/01/2020
Agnostic Learning of Halfspaces with Gradient Descent via Soft Margins
We analyze the properties of gradient descent on convex surrogates for t...
research
∙
05/29/2020
Agnostic Learning of a Single Neuron with Gradient Descent
We consider the problem of learning the best-fitting single neuron as me...
research
∙
10/07/2019