research
∙
05/28/2022
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
Sparsely activated transformers, such as Mixture of Experts (MoE), have ...
share
research
∙
11/03/2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task
This report describes Microsoft's machine translation systems for the WM...
research
∙
09/22/2021
Scalable and Efficient MoE Training for Multitask Multilingual Models
The Mixture of Experts (MoE) models are an emerging class of sparsely ac...
research
∙
09/10/2021
Improving Multilingual Translation by Representation and Gradient Regularization
Multilingual Neural Machine Translation (NMT) enables one model to serve...
research
∙
06/25/2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
While pretrained encoders have achieved success in various natural langu...
research
∙
12/31/2020