research
∙
06/24/2023
Is Pre-training Truly Better Than Meta-Learning?
In the context of few-shot learning, it is currently believed that a fix...
share
research
∙
06/24/2023
Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data
Current trends to pre-train capable Large Language Models (LLMs) mostly ...
research
∙
04/28/2023
Are Emergent Abilities of Large Language Models a Mirage?
Recent work claims that large language models display emergent abilities...
research
∙
03/15/2023
Transformer Models for Type Inference in the Simply Typed Lambda Calculus: A Case Study in Deep Learning for Code
Despite a growing body of work at the intersection of deep learning and ...
research
∙
08/02/2022
The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence
Recently, it has been observed that a transfer learning solution might b...
research
∙
12/24/2021
Does MAML Only Work via Feature Re-use? A Data Centric Perspective
Recent work has suggested that a good embedding is all we need to solve ...
research
∙
12/24/2021
The Curse of Zero Task Diversity: On the Failure of Transfer Learning to Outperform MAML and their Empirical Equivalence
Recently, it has been observed that a transfer learning solution might b...
research
∙
03/12/2019
Theory III: Dynamics and Generalization in Deep Networks
We review recent observations on the dynamical systems induced by gradie...
research
∙
07/25/2018
A Surprising Linear Relationship Predicts Test Performance in Deep Networks
Given two networks with the same training loss on a dataset, when would ...
research
∙
06/29/2018
Theory IIIb: Generalization in Deep Networks
A main puzzle of deep neural networks (DNNs) revolves around the apparen...
research
∙
01/07/2018
Theory of Deep Learning IIb: Optimization Properties of SGD
In Theory IIb we characterize with a mix of theory and experiments the o...
research
∙
12/30/2017