research
∙
06/05/2023
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Model pre-training on large text corpora has been demonstrated effective...
share
research
∙
03/10/2023
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Contrastive loss has been increasingly used in learning representations ...
research
∙
12/10/2022
SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
The mixture of Expert (MoE) parallelism is a recent advancement that sca...
research
∙
06/22/2022
Efficient and effective training of language and graph neural network models
Can we combine heterogenous graph structure with text to learn high-qual...
research
∙
06/07/2022
DynaMaR: Dynamic Prompt with Mask Token Representation
Recent research has shown that large language models pretrained using un...
research
∙
04/30/2022
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud
Existing general purpose frameworks for gigantic model training, i.e., m...
research
∙
02/28/2022
Multi-modal Alignment using Representation Codebook
Aligning signals from different modalities is an important step in visio...
research
∙
02/21/2022
Vision-Language Pre-Training with Triple Contrastive Learning
Vision-language representation learning largely benefits from image-text...
research
∙
10/30/2021
Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
Pre-training and then fine-tuning large language models is commonly used...
research
∙
09/24/2021
MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling
Vision-and-Language Pre-training (VLP) improves model performance for do...
research
∙
05/16/2020