research
∙
06/01/2023
Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior
Pre-trained machine learning (ML) models have shown great performance fo...
share
research
∙
06/22/2022
GACT: Activation Compressed Training for General Architectures
Training large neural network (NN) models requires extensive memory reso...
research
∙
02/24/2022
AutoIP: A United Framework to Integrate Physics into Gaussian Processes
Physics modeling is critical for modern science and engineering applicat...
research
∙
05/16/2021
LocalNewton: Reducing Communication Bottleneck for Distributed Learning
To address the communication bottleneck problem in distributed optimizat...
research
∙
03/17/2020
Rethinking Batch Normalization in Transformers
The standard normalization method for neural network (NN) models used in...
research
∙
12/16/2019
PyHessian: Neural Networks Through the Lens of the Hessian
We present PyHessian, a new scalable framework that enables fast computa...
research
∙
06/10/2019
ANODEV2: A Coupled Neural ODE Evolution Framework
It has been observed that residual networks can be viewed as the explici...
research
∙
12/16/2018
Trust Region Based Adversarial Attack on Neural Networks
Deep Neural Networks are quite vulnerable to adversarial perturbations. ...
research
∙
12/04/2018
Parameter Re-Initialization through Cyclical Batch Size Schedules
Optimal parameter initialization remains a crucial problem for neural ne...
research
∙
10/02/2018
Large batch size training of neural networks with adversarial training and second-order information
Stochastic Gradient Descent (SGD) methods using randomly selected batche...
research
∙
05/25/2015
Statistical and Algorithmic Perspectives on Randomized Sketching for Ordinary Least-Squares -- ICML
We consider statistical and algorithmic aspects of solving large-scale l...
research
∙
12/29/2014
Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels
We consider the problem of improving the efficiency of randomized Fourie...
research
∙
06/23/2014