research
∙
07/11/2022
Exploring Length Generalization in Large Language Models
The ability to extrapolate from short problem instances to longer ones i...
share
research
∙
06/29/2022
Solving Quantitative Reasoning Problems with Language Models
Language models have achieved remarkable performance on a wide range of ...
research
∙
03/11/2022
Block-Recurrent Transformers
We introduce the Block-Recurrent Transformer, which applies a transforme...
research
∙
02/12/2021
Explaining Neural Scaling Laws
The test loss of well-trained neural networks often follows precise powe...
research
∙
12/05/2020
When Do Curricula Work?
Inspired by human learning, researchers have proposed ordering examples ...
research
∙
08/19/2020
Asymptotics of Wide Convolutional Neural Networks
Wide neural networks have proven to be a rich class of architectures for...
research
∙
08/17/2020
Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible
Machine learning is predicated on the concept of generalization: a model...
research
∙
07/14/2020
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
A central challenge in developing versatile machine learning systems is ...
research
∙
03/04/2020
The large learning rate phase of deep learning: the catapult mechanism
The choice of initial learning rate can have a profound effect on the pe...
research
∙
02/20/2020
Affinity and Diversity: Quantifying Mechanisms of Data Augmentation
Though data augmentation has become a standard component of deep neural ...
research
∙
09/25/2019
Asymptotics of Wide Networks from Feynman Diagrams
Understanding the asymptotic behavior of wide networks is of considerabl...
research
∙
12/12/2018