research
∙
06/29/2023
Modeling Parallel Programs using Large Language Models
Parallel software codes in high performance computing (HPC) continue to ...
share
research
∙
06/19/2023
Pipit: Enabling programmatic analysis of parallel execution traces
Performance analysis is an important part of the oft-repeated, iterative...
research
∙
05/22/2023
Communication-minimizing Asynchronous Tensor Parallelism
As state-of-the-art neural networks scale to billions of parameters, des...
research
∙
03/11/2023
A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training
Mixture-of-Experts (MoE) is a neural network architecture that adds spar...
research
∙
02/10/2023
Exploiting Sparsity in Pruned Neural Networks to Optimize Large Model Training
Parallel training of neural networks at scale is challenging due to sign...
research
∙
05/09/2022
Designing an Interactive, Notebook-Embedded, Tree Visualization to Support Exploratory Performance Analysis
Interactive visualization via direct manipulation has inherent design tr...
research
∙
11/09/2021
How to Train Your Neural Network: A Comparative Evaluation
The field of deep learning has witnessed a remarkable shift towards extr...
research
∙
10/25/2021
AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning
In the last few years, the memory requirements to train state-of-the-art...
research
∙
07/07/2020
Analytics of Longitudinal System Monitoring Data for Performance Prediction
In recent years, several HPC facilities have started continuous monitori...
research
∙
07/01/2020