research
∙
07/21/2023
On the Universality of Linear Recurrences Followed by Nonlinear Projections
In this note (work in progress towards a full-length paper) we show that...
share
research
∙
03/16/2023
Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning
In contrast to the natural capabilities of humans to learn new tasks in ...
research
∙
03/11/2023
Resurrecting Recurrent Neural Networks for Long Sequences
Recurrent Neural Networks (RNNs) offer fast inference on long sequences ...
research
∙
01/19/2023
An SDE for Modeling SAM: Theory and Insights
We study the SAM (Sharpness-Aware Minimization) optimizer which has rece...
research
∙
09/19/2022
On the Theoretical Properties of Noise Correlation in Stochastic Optimization
Studying the properties of stochastic noise to optimize complex non-conv...
research
∙
06/09/2022
Explicit Regularization in Overparametrized Models via Noise Injection
Injecting noise within gradient descent has several desirable features. ...
research
∙
06/07/2022
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
Transformers have achieved remarkable success in several domains, rangin...
research
∙
02/06/2022
Anticorrelated Noise Injection for Improved Generalization
Injecting artificial noise into gradient descent (GD) is commonly employ...
research
∙
01/02/2022
Randomized Signature Layers for Signal Extraction in Time Series Data
Time series analysis is a widespread task in Natural Sciences, Social Sc...
research
∙
12/10/2021
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity
Gradient descent ascent (GDA), the simplest single-loop algorithm for no...
research
∙
10/25/2021
On the Second-order Convergence Properties of Random Search Methods
We study the theoretical convergence properties of random-search methods...
research
∙
02/23/2021
Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization
Viewing optimization methods as numerical integrators for ordinary diffe...
research
∙
09/01/2020
Learning explanations that are hard to vary
In this paper, we investigate the principle that `good explanations are ...
research
∙
07/07/2020
An Accelerated DFO Algorithm for Finite-sum Convex Functions
Derivative-free optimization (DFO) has recently gained a lot of momentum...
research
∙
11/12/2019
Shadowing Properties of Optimization Algorithms
Ordinary differential equation (ODE) models of gradient-based optimizati...
research
∙
07/02/2019
The Role of Memory in Stochastic Optimization
The choice of how to retain information about past gradients dramaticall...
research
∙
10/05/2018