research
∙
07/07/2021
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly availabl...
share
research
∙
02/24/2021
Zero-Shot Text-to-Image Generation
Text-to-image generation has traditionally focused on finding better mod...
research
∙
10/28/2020
Scaling Laws for Autoregressive Generative Modeling
We identify empirical scaling laws for the cross-entropy loss in four do...
research
∙
05/28/2020
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and ben...
research
∙
01/23/2020
Scaling Laws for Neural Language Models
We study empirical scaling laws for language model performance on the cr...
research
∙
12/13/2019
Dota 2 with Large Scale Deep Reinforcement Learning
On April 13th, 2019, OpenAI Five became the first AI system to defeat th...
research
∙
04/23/2019
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory t...
research
∙
01/14/2019
FoundationDB Record Layer: A Multi-Tenant Structured Datastore
The FoundationDB Record Layer is an open source library that provides a ...
research
∙
11/06/2017
Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks
Deep neural networks are commonly developed and trained in 32-bit floati...
research
∙
09/30/2015