Zachary Nado | DeepAI

DeepAI AI Chat

Featured Co-authors

Fei Wang
209 publications
Yarin Gal
126 publications
Zoubin Ghahramani
90 publications
Philipp Hennig
82 publications
Andrea Montanari
78 publications
Orhan Firat
68 publications
Jie Ren
66 publications
Kevin Murphy
62 publications
Roger Grosse
56 publications
Zi Wang
56 publications
Jacob Eisenstein
54 publications

∙ 06/12/2023

Benchmarking Neural Network Training Algorithms

Training algorithms, broadly construed, are an essential part of every d...

6 George E. Dahl, et al. ∙

∙ 03/09/2023

Kernel Regression with Infinite-Width Neural Networks on Millions of Examples

Neural kernels have drastically increased performance on diverse and non...

0 Ben Adlam, et al. ∙

∙ 11/23/2022

Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks

Bayesian deep learning seeks to equip deep neural networks with the abil...

0 Neil Band, et al. ∙

∙ 07/29/2022

Adaptive Gradient Methods at the Edge of Stability

Very little is known about the training dynamics of adaptive gradient me...

30 Jeremy M Cohen, et al. ∙

∙ 07/07/2022

Pre-training helps Bayesian optimization too

Bayesian optimization (BO) has become a popular strategy for global opti...

17 Zi Wang, et al. ∙

∙ 12/15/2021

Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach

Black box optimization requires specifying a search space to explore for...

0 Setareh Ariafar, et al. ∙

∙ 10/08/2021

A Loss Curvature Perspective on Training Instability in Deep Learning

In this work, we study the evolution of the loss Hessian across many cla...

47 Justin Gilmer, et al. ∙

∙ 06/07/2021

Uncertainty Baselines: Benchmarks for Uncertainty Robustness in Deep Learning

High-quality estimates of uncertainty and robustness are crucial for num...

0 Zachary Nado, et al. ∙

∙ 02/12/2021

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes

Recently the LARS and LAMB optimizers have been proposed for training ne...

7 Zachary Nado, et al. ∙

∙ 11/06/2020

Underspecification Presents Challenges for Credibility in Modern Machine Learning

ML models often exhibit unexpectedly poor behavior when they are deploye...

30 Alexander D'Amour, et al. ∙

∙ 07/10/2020

Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks

Accurate estimation of predictive uncertainty in modern neural networks ...

20 Shreyas Padhy, et al. ∙

∙ 06/19/2020

Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift

Covariate shift has been shown to sharply degrade both predictive accura...

10 Zachary Nado, et al. ∙

∙ 10/11/2019

On Empirical Comparisons of Optimizers for Deep Learning

Selecting an optimizer is a central step in the contemporary deep learni...

0 Dami Choi, et al. ∙

∙ 07/09/2019

Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model

Increasing the batch size is a popular way to speed up neural network tr...

3 Guodong Zhang, et al. ∙

∙ 06/06/2019

Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

Modern machine learning methods including deep learning have achieved gr...

9 Yaniv Ovadia, et al. ∙

∙ 10/16/2018

AutoGraph: Imperative-style Coding with Graph-based Performance

There is a perceived trade-off between machine learning code that is eas...

0 Dan Moldovan, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world