research
∙
06/18/2023
DropCompute: simple and more robust distributed synchronous training via compute variance reduction
Background: Distributed training is essential for large scale training o...
share
research
∙
02/06/2022
Energy awareness in low precision neural networks
Power consumption is a major obstacle in the deployment of deep neural n...
research
∙
12/19/2021
Logarithmic Unbiased Quantization: Practical 4-bit Training in Deep Learning
Quantization of the weights and activations is one of the main methods t...
research
∙
10/01/2020
Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
Background: Catastrophic forgetting is the notorious vulnerability of ne...
research
∙
06/15/2020
Neural gradients are lognormally distributed: understanding sparse and quantized training
Neural gradient compression remains a main bottleneck in improving train...
research
∙
12/03/2019
The Knowledge Within: Methods for Data-Free Model Compression
Background: Recently, an extensive amount of research has been focused o...
research
∙
09/26/2019
At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?
Background: Recent developments have made it possible to accelerate neur...
research
∙
01/27/2019
Augment your batch: better training with larger batches
Large-batch SGD is important for scaling training of deep neural network...
research
∙
10/02/2018
ACIQ: Analytical Clipping for Integer Quantization of neural networks
Unlike traditional approaches that focus on the quantization at the netw...
research
∙
05/25/2018
Scalable Methods for 8-bit Training of Neural Networks
Quantized Neural Networks (QNNs) are often used to improve network effic...
research
∙
03/27/2018
Bayesian Gradient Descent: Online Variational Bayes Learning with Increased Robustness to Catastrophic Forgetting and Weight Pruning
We suggest a novel approach for the estimation of the posterior distribu...
research
∙
03/05/2018
Norm matters: efficient and accurate normalization schemes in deep networks
Over the past few years batch-normalization has been commonly used in de...
research
∙
02/14/2018
On the Blindspots of Convolutional Networks
Deep convolutional network has been the state-of-the-art approach for a ...
research
∙
01/14/2018
Fix your classifier: the marginal value of training the last weight layer
Neural networks are commonly used as models for classification for a wid...
research
∙
10/27/2017
The Implicit Bias of Gradient Descent on Separable Data
We show that gradient descent on an unregularized logistic regression pr...
research
∙
05/24/2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks
Background: Deep learning models are typically trained using stochastic ...
research
∙
02/19/2017
Exponentially vanishing sub-optimal local minima in multilayer neural networks
Background: Statistical mechanics results (Dauphin et al. (2014); Chorom...
research
∙
10/02/2016
Deep unsupervised learning through spatial contrasting
Convolutional networks have marked their place over the last few years a...
research
∙
12/20/2014