research
∙
09/11/2023
Textbooks Are All You Need II: phi-1.5 technical report
We continue the investigation into the power of smaller Transformer-base...
share
research
∙
09/01/2023
Efficient RLHF: Reducing the Memory Usage of PPO
Reinforcement Learning with Human Feedback (RLHF) has revolutionized lan...
research
∙
06/20/2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks
We study the implicit bias of batch normalization trained by gradient de...
research
∙
05/03/2023
Plan, Eliminate, and Track – Language Models are Good Teachers for Embodied Agents
Pre-trained large language models (LLMs) capture procedural knowledge ab...
research
∙
04/07/2023
On the Importance of Contrastive Loss in Multimodal Learning
Recently, contrastive learning approaches (e.g., CLIP (Radford et al., 2...
research
∙
03/22/2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Artificial intelligence (AI) researchers have been developing and refini...
research
∙
09/22/2022
Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
We provide theoretical convergence guarantees for score-based generative...
research
∙
08/04/2022
Towards Understanding Mixture of Experts in Deep Learning
The Mixture-of-Experts (MoE) layer, a sparsely-activated model controlle...
research
∙
07/13/2022
Towards understanding how momentum improves generalization in deep learning
Stochastic gradient descent (SGD) with momentum is widely used for train...
research
∙
05/31/2022
Learning (Very) Simple Generative Models Is Hard
Motivated by the recent empirical successes of deep generative models, w...
research
∙
06/17/2021
LoRA: Low-Rank Adaptation of Large Language Models
The dominant paradigm of natural language processing consists of large-s...
research
∙
01/01/2021
When Is Generalizable Reinforcement Learning Tractable?
Agents trained by reinforcement learning (RL) often fail to generalize b...
research
∙
02/04/2019
Can SGD Learn Recurrent Neural Networks with Provable Generalization?
Recurrent Neural Networks (RNNs) are among the most popular models in se...
research
∙
11/12/2018
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
Neural networks have great success in many machine learning applications...
research
∙
11/17/2017
Neon2: Finding Local Minima via First-Order Oracles
We propose a reduction for non-convex optimization that can (1) turn a s...
research
∙
01/14/2016
Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Word embeddings are ubiquitous in NLP and information retrieval, but it'...
research
∙
02/12/2015
