research
∙
06/12/2023
Benchmarking Neural Network Training Algorithms
Training algorithms, broadly construed, are an essential part of every d...
share
research
∙
07/29/2022
Adaptive Gradient Methods at the Edge of Stability
Very little is known about the training dynamics of adaptive gradient me...
research
∙
10/06/2020
A Unifying View on Implicit Bias in Training Linear Neural Networks
We study the implicit bias of gradient flow (i.e., gradient descent with...
research
∙
03/16/2020
Explaining Memorization and Generalization: A Large-Scale Study with Coherent Gradients
Coherent Gradients is a recently proposed hypothesis to explain why over...
research
∙
11/21/2019
Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks
Batch Normalization (BN) is a highly successful and widely used batch de...
research
∙
01/29/2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
To understand the dynamics of optimization in deep neural networks, we d...
research
∙
12/08/2017
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks
Progress in deep learning is slowed by the days or weeks it takes to tra...
research
∙
07/26/2012