research
∙
05/15/2023
ReLU soothes the NTK condition number and accelerates optimization for wide neural networks
Rectified linear unit (ReLU), as a non-linear activation function, is we...
share
research
∙
02/08/2023
Cut your Losses with Squentropy
Nearly all practical neural models for classification are trained using ...
research
∙
02/17/2022
Limitations of Neural Collapse for Understanding Generalization in Deep Learning
The recent work of Papyan, Han, Donoho (2020) presented an intriguin...
research
∙
06/12/2020
Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks
Modern neural architectures for classification tasks are trained using t...
research
∙
11/06/2018