research
∙
05/31/2023
Adam Accumulation to Reduce Memory Footprints of both Activations and Gradients for Large-scale DNN Training
Running out of GPU memory has become a main bottleneck for large-scale D...
share
research
∙
01/21/2023
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction
With the growing model size, deep neural networks (DNN) are increasingly...
research
∙
12/29/2021
Dense-to-Sparse Gate for Mixture-of-Experts
Mixture-of-experts (MoE) is becoming popular due to its success in impro...
research
∙
07/19/2021
ZIPPER: Exploiting Tile- and Operator-level Parallelism for General and Scalable Graph Neural Network Acceleration
Graph neural networks (GNNs) start to gain momentum after showing signif...
research
∙
05/12/2021
Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads
Recent trend towards increasing large machine learning models require bo...
research
∙
03/14/2021
CrossoverScheduler: Overlapping Multiple Distributed Training Applications in a Crossover Manner
Distributed deep learning workloads include throughput-intensive trainin...
research
∙
09/02/2020
Architectural Implications of Graph Neural Networks
Graph neural networks (GNN) represent an emerging line of deep learning ...
research
∙
10/19/2018
Towards Efficient Large-Scale Graph Neural Network Computing
Recent deep learning models have moved beyond low-dimensional regular gr...
research
∙
05/22/2018