research
∙
02/17/2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon
The Sharpness Aware Minimization (SAM) optimization algorithm has been s...
share
research
∙
10/10/2022
Second-order regression models exhibit progressive sharpening to the edge of stability
Recent studies of gradient descent with large step sizes have shown that...
research
∙
07/19/2022
Deep equilibrium networks are sensitive to initialization statistics
Deep equilibrium networks (DEQs) are a promising way to construct models...
research
∙
03/29/2021
One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks
Can deep learning solve multiple tasks simultaneously, even when they ar...
research
∙
10/14/2020
Temperature check: theory and practice for training models with softmax-cross-entropy losses
The softmax function combined with a cross-entropy loss is a principled ...
research
∙
05/15/2020