research
∙
07/14/2023
Fast Algorithms for a New Relaxation of Optimal Transport
We introduce a new class of objectives for optimal transport computation...
share
research
∙
06/24/2023
H_2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Large Language Models (LLMs), despite their recent impressive accomplish...
research
∙
06/20/2023
InRank: Incremental Low-Rank Learning
The theory of greedy low-rank learning (GLRL) aims to explain the impres...
research
∙
05/25/2023
Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer
Transformer architecture has shown impressive performance in multiple re...
research
∙
05/17/2023
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt
Large Language Models (LLMs), armed with billions of parameters, exhibit...
research
∙
03/13/2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
The high computational and memory requirements of large language model (...
research
∙
06/02/2022
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees
Communication compression is a crucial technique for modern distributed ...
research
∙
06/02/2022
Decentralized Training of Foundation Models in Heterogeneous Environments
Training foundation models, such as GPT-3 and PaLM, can be extremely exp...
research
∙
11/30/2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
Overparameterized neural networks generalize well but are expensive to t...
research
∙
11/12/2021
Satellite Images and Deep Learning to Identify Discrepancy in Mailing Addresses with Applications to Census 2020 in Houston
The accuracy and completeness of population estimation would significant...
research
∙
10/28/2021
Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
Recent advances in efficient Transformers have exploited either the spar...
research
∙
12/31/2020
A Constant-time Adaptive Negative Sampling
Softmax classifiers with a very large number of classes naturally occur ...
research
∙
08/30/2020
SOLAR: Sparse Orthogonal Learned and Random Embeddings
Dense embedding models are commonly deployed in commercial search engine...
research
∙
07/02/2020
Climbing the WOL: Training for Cheaper Inference
Efficient inference for wide output layers (WOLs) is an essential yet ch...
research
∙
12/04/2019
Angular Visual Hardness
Although convolutional neural networks (CNNs) are inspired by the mechan...
research
∙
10/30/2019
Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation
Stochastic Gradient Descent or SGD is the most popular optimization algo...
research
∙
03/07/2019
SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems
Deep Learning (DL) algorithms are the central focus of modern machine le...
research
∙
10/07/2017
Unique Entity Estimation with Application to the Syrian Conflict
Entity resolution identifies and removes duplicate entities in large, no...
research
∙
12/06/2016