research
∙
06/25/2023
Mirage: Towards Low-interruption Services on Batch GPU Clusters with Reinforcement Learning
Accommodating long-running deep learning (DL) training and inference job...
share
research
∙
05/02/2023
F2: Designing a Key-Value Store for Large Skewed Workloads
Today's key-value stores are either disk-optimized, focusing on large da...
research
∙
01/06/2023
Does compressing activations help model parallel training?
Large-scale Transformer models are known for their exceptional performan...
research
∙
09/30/2022
Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning
Dynamic adaptation has become an essential technique in accelerating dis...
research
∙
08/23/2022
Not All GPUs Are Created Equal: Characterizing Variability in Large-Scale, Accelerator-Rich Systems
Scientists are increasingly exploring and utilizing the massive parallel...
research
∙
03/10/2022
LlamaTune: Sample-Efficient DBMS Configuration Tuning
Tuning a database system to achieve optimal performance on a given workl...
research
∙
02/24/2022
BagPipe: Accelerating Deep Recommendation Model Training
Deep learning based recommendation models (DLRM) are widely used in seve...
research
∙
02/04/2022
Marius++: Large-Scale Training of Graph Neural Networks on a Single Machine
Graph Neural Networks (GNNs) have emerged as a powerful model for ML ove...
research
∙
11/20/2021
Doing More by Doing Less: How Structured Partial Backpropagation Improves Deep Learning Clusters
Many organizations employ compute clusters equipped with accelerators su...
research
∙
07/04/2021
KAISA: An Adaptive Second-order Optimizer Framework for Deep Neural Networks
Kronecker-factored Approximate Curvature (K-FAC) has recently been shown...
research
∙
02/28/2021
On the Utility of Gradient Compression in Distributed Training Systems
Rapid growth in data sets and the scale of neural network architectures ...
research
∙
02/02/2021
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning
With the rapid adoption of machine learning (ML), a number of domains no...
research
∙
01/20/2021
Marius: Learning Massive Graph Embeddings on a Single Machine
We propose a new framework for computing the embeddings of large-scale g...
research
∙
01/18/2021
Accelerating Deep Learning Inference via Learned Caches
Deep Neural Networks (DNNs) are witnessing increased adoption in multipl...
research
∙
10/29/2020
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification
Distributed model training suffers from communication bottlenecks due to...
research
∙
10/06/2020
Move Fast and Meet Deadlines: Fine-grained Real-time Stream Processing with Cameo
Resource provisioning in multi-tenant stream processing systems faces th...
research
∙
02/07/2020
Accelerating Deep Learning Inference via Freezing
Over the last few years, Deep Neural Networks (DNNs) have become ubiquit...
research
∙
11/22/2019
Archipelago: A Scalable Low-Latency Serverless Platform
The increased use of micro-services to build web applications has spurre...
research
∙
10/11/2019
Blink: Fast and Generic Collectives for Distributed ML
Model parameter synchronization across GPUs introduces high overheads fo...
research
∙
07/02/2019
Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads
Modern distributed machine learning (ML) training workloads benefit sign...
research
∙
05/02/2019
Parity Models: A General Framework for Coding-Based Resilience in ML Inference
Machine learning models are becoming the primary workhorses for many app...
research
∙
03/29/2019
SysML: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
research
∙
01/17/2019
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads
With widespread advances in machine learning, a number of large enterpri...
research
∙
10/23/2018
numpywren: serverless linear algebra
Linear algebra operations are widely used in scientific computing and ma...
research
∙
06/04/2018
Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation
Machine learning algorithms are typically run on large scale, distribute...
research
∙
02/20/2017
Hemingway: Modeling Distributed Optimization Algorithms
Distributed optimization algorithms are widely used in many industrial m...
research
∙
02/17/2016
Large Scale Kernel Learning using Block Coordinate Descent
We demonstrate that distributed block coordinate descent can quickly sol...
research
∙
05/26/2015