research
∙
04/02/2023
Saddle-to-Saddle Dynamics in Diagonal Linear Networks
In this paper we fully describe the trajectory of gradient flow over dia...
share
research
∙
02/17/2023
(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
In this paper, we investigate the impact of stochasticity and large step...
research
∙
06/17/2021
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Understanding the implicit bias of training algorithms is of crucial imp...
research
∙
07/01/2020
On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Constant step-size Stochastic Gradient Descent exhibits two phases: a tr...
research
∙
07/01/2020