research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
share
research
∙
01/28/2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Pretrained general-purpose language models can achieve state-of-the-art ...
research
∙
11/20/2020
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
We present a hierarchical VAE that, for the first time, outperforms the ...
research
∙
05/28/2020
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and ben...
research
∙
01/23/2020
Scaling Laws for Neural Language Models
We study empirical scaling laws for language model performance on the cr...
research
∙
04/23/2019
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory t...
research
∙
07/24/2017
Exploring Neural Transducers for End-to-End Speech Recognition
In this work, we perform an empirical comparison among the CTC, RNN-Tran...
research
∙
05/11/2017
Reducing Bias in Production Speech Models
Replacing hand-engineered pipelines with end-to-end deep learning system...
research
∙
03/15/2017
Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting
Keyword spotting (KWS) constitutes a major component of human-technology...
research
∙
12/10/2016