Zhiquan Lai | DeepAI

AI Chat

Featured Co-authors

Dongsheng Li
104 publications
Ning Liu
46 publications
Yiming Zhang
42 publications
Yuliang Liu
36 publications
Weijie Liu
19 publications
Linbo Qiao
13 publications
Kai Lu
13 publications
Chenyu Wang
11 publications
Yufan Zhang
7 publications
Yu Tang
6 publications
Xingcheng Zhang
5 publications

∙ 05/25/2023

Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training

Deep learning is experiencing a rise in foundation models that are expec...

0 Shengwei Li, et al. ∙

∙ 06/10/2022

Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models

Foundation models are becoming the dominant deep learning technologies. ...

0 Zhiquan Lai, et al. ∙

∙ 03/30/2022

DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation

The further development of deep neural networks is hampered by the limit...

0 Yu Tang, et al. ∙

∙ 10/18/2021

EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks

Distributed data-parallel training has been widely used for natural lang...

0 Shengwei Li, et al. ∙

∙ 10/05/2021

S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning

Distributed stochastic gradient descent (SGD) approach has been widely u...

0 Keshi Ge, et al. ∙

∙ 04/13/2021

Hierarchical Adaptive Pooling by Capturing High-order Dependency for Graph Representation Learning

Graph neural networks (GNN) have been proven to be mature enough for han...

6 Ning Liu, et al. ∙

∙ 06/10/2020

ADMMiRNN: Training RNN with Stable Convergence via An Efficient ADMM Approach

It is hard to train Recurrent Neural Network (RNN) with stable convergen...

0 Yu Tang, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world