research
∙
09/20/2023
Kosmos-2.5: A Multimodal Literate Model
We present Kosmos-2.5, a multimodal literate model for machine reading o...
share
research
∙
07/05/2023
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Scaling sequence length has become a critical demand in the era of large...
research
∙
11/23/2022
TorchScale: Transformers at Scale
Large Transformers have achieved state-of-the-art performance across man...
research
∙
09/21/2022
DPCN: Towards Deadline-aware Payment Channel Networks
Payment channel is a class of techniques designed to solve the scalabili...
research
∙
08/22/2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
A big convergence of language, vision, and multimodal pretraining is eme...
research
∙
06/13/2022
Language Models are General-Purpose Interfaces
Foundation models have received much attention due to their effectivenes...
research
∙
02/28/2020
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
We propose to pre-train a unified language model for both autoencoding a...
research
∙
02/25/2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
Pre-trained language models (e.g., BERT (Devlin et al., 2018) and its va...
research
∙
05/08/2019
