b'Daniel M. Roy'

research

∙ 06/30/2023

The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit

In deep learning theory, the covariance matrix of the representations se...

0 Lorenzo Noci, et al. ∙

research

∙ 12/27/2022

Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization

To date, no "information-theoretic" frameworks for reasoning about gener...

8 Mahdi Haghifam, et al. ∙

research

∙ 12/26/2022

Statistical minimax theorems via nonstandard analysis

For statistical decision problems with finite parameter space, it is wel...

0 Haosui Duanmu, et al. ∙

research

∙ 10/25/2022

Pruning's Effect on Generalization Through the Lens of Training and Regularization

Practitioners frequently observe that pruning improves model generalizat...

0 Tian Jin, et al. ∙

research

∙ 07/25/2022

Statistical Inference with Stochastic Gradient Algorithms

Tuning of stochastic gradient algorithms (SGAs) for optimization and sam...

0 Jeffrey Negrea, et al. ∙

research

∙ 06/29/2022

Understanding Generalization via Leave-One-Out Conditional Mutual Information

We study the mutual information between (certain summaries of) the outpu...

0 Mahdi Haghifam, et al. ∙

research

∙ 06/06/2022

The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization

The logit outputs of a feedforward neural network at initialization are ...

0 Mufan Bill Li, et al. ∙

research

∙ 02/10/2022

Adaptively Exploiting d-Separators with Causal Bandits

Multi-armed bandit problems provide a framework to identify the optimal ...

0 Blair Bilodeau, et al. ∙

research

∙ 12/28/2021

Admissibility is Bayes optimality with infinitesimals

We give an exact characterization of admissibility in statistical decisi...

0 Haosui Duanmu, et al. ∙

research

∙ 11/09/2021

Towards a Unified Information-Theoretic Framework for Generalization

In this work, we investigate the expressiveness of the "conditional mutu...

0 Mahdi Haghifam, et al. ∙

research

∙ 09/21/2021

Minimax Rates for Conditional Density Estimation via Empirical Entropy

We consider the task of estimating a conditional density using i.i.d. sa...

0 Blair Bilodeau, et al. ∙

research

∙ 06/07/2021

The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization

Theoretical results show that neural networks can be approximated by Gau...

0 Mufan Bill Li, et al. ∙

research

∙ 04/28/2021

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

As the size and complexity of models and datasets grow, so does the need...

0 Ali Ramezani-Kebrya, et al. ∙

research

∙ 12/14/2020

NeurIPS 2020 Competition: Predicting Generalization in Deep Learning

Understanding generalization in deep learning is arguably one of the mos...

0 Yiding Jiang, et al. ∙

research

∙ 11/07/2020

Existence of matching priors on compact spaces

A matching prior at level 1-α is a prior such that an associated 1-α cre...

0 Haosui Duanmu, et al. ∙

research

∙ 11/05/2020

On the Information Complexity of Proper Learners for VC Classes in the Realizable Case

We provide a negative resolution to a conjecture of Steinke and Zakynthi...

0 Mahdi Haghifam, et al. ∙

research

∙ 10/28/2020

Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel

In suitably initialized wide networks, small learning rates transform de...

0 Stanislav Fort, et al. ∙

research

∙ 10/26/2020

Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability

To date, there has been no formal study of the statistical cost of inter...

0 Gintare Karolina Dziugaite, et al. ∙

research

∙ 10/22/2020

In Search of Robust Measures of Generalization

One of the principal scientific challenges in deep learning is explainin...

9 Gintare Karolina Dziugaite, et al. ∙

research

∙ 09/18/2020

Pruning Neural Networks at Initialization: Why are We Missing the Mark?

Recent work has explored the possibility of pruning neural networks at i...

9 Jonathan Frankle, et al. ∙

research

∙ 07/13/2020

Relaxing the I.I.D. Assumption: Adaptive Minimax Optimal Sequential Prediction with Expert Advice

We consider sequential prediction with expert advice when the data are g...

0 Blair Bilodeau, et al. ∙

research

∙ 07/02/2020

Improved Bounds on Minimax Regret under Logarithmic Loss via Self-Concordance

We consider the classical problem of sequential probability assignment u...

0 Blair Bilodeau, et al. ∙

research

∙ 06/19/2020

On the role of data in PAC-Bayes bounds

The dominant term in PAC-Bayes bounds is often the Kullback–Leibler dive...

9 Gintare Karolina Dziugaite, et al. ∙

research

∙ 04/27/2020

Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms

The information-theoretic framework of Russo and J. Zou (2016) and Xu an...

25 Mahdi Haghifam, et al. ∙

research

∙ 12/14/2019

Approximations in Probabilistic Programs

We study the first-order probabilistic programming language introduced b...

0 Ekansh Sharma, et al. ∙

research

∙ 12/11/2019

Linear Mode Connectivity and the Lottery Ticket Hypothesis

We introduce "instability analysis," a framework for assessing whether t...

13 Jonathan Frankle, et al. ∙

research

∙ 12/09/2019

In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictors

We propose to study the generalization error of a learned predictor ĥ in...

11 Jeffrey Negrea, et al. ∙

research

∙ 11/06/2019

Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates

In this work, we improve upon the stepwise analysis of noisy iterative l...

15 Jeffrey Negrea, et al. ∙

research

∙ 08/20/2019

Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes

The developments of Rademacher complexity and PAC-Bayesian theory have b...

4 Jun Yang, et al. ∙

research

∙ 08/17/2019

Black-box constructions for exchangeable sequences of random multisets

We develop constructions for exchangeable sequences of point processes t...

2 Creighton Heaukulani, et al. ∙

research

∙ 08/16/2019

NUQSGD: Improved Communication Efficiency for Data-parallel SGD via Nonuniform Quantization

As the size and complexity of models and datasets grow, so does the need...

4 Ali Ramezani-Kebrya, et al. ∙

research

∙ 03/05/2019

The Lottery Ticket Hypothesis at Scale

Recent work on the "lottery ticket hypothesis" proposes that randomly-in...

8 Jonathan Frankle, et al. ∙

research

∙ 02/26/2018

The Beta-Bernoulli process and algebraic effects

In this paper we analyze the Beta-Bernoulli process from Bayesian nonpar...

0 Sam Staton, et al. ∙

research

∙ 02/26/2018

Data-dependent PAC-Bayes priors via differential privacy

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 19...

0 Gintare Karolina Dziugaite, et al. ∙

research

∙ 01/31/2018

On the computability of graphons

We investigate the relative computability of exchangeable binary relatio...

0 Nathanael L. Ackerman, et al. ∙

research

∙ 12/26/2017

Entropy-SGD optimizes the prior of a PAC-Bayes bound: Data-dependent PAC-Bayes priors via differential privacy

We show that Entropy-SGD (Chaudhari et al., 2016), when viewed as a lear...

0 Gintare Karolina Dziugaite, et al. ∙

research

∙ 12/06/2017

Exchangeable modelling of relational data: checking sparsity, train-test splitting, and sparse exchangeable Poisson matrix factorization

A variety of machine learning tasks---e.g., matrix factorization, topic ...

0 Victor Veitch, et al. ∙

research

∙ 12/05/2017

An estimator for the tail-index of graphex processes

Sparse exchangeable graphs resolve some pathologies in traditional rando...

0 Zacharie Naulet, et al. ∙

research

∙ 08/02/2016

A study of the effect of JPG compression on adversarial images

Neural network image classifiers are known to be vulnerable to adversari...

0 Gintare Karolina Dziugaite, et al. ∙

research

∙ 07/07/2016

A characterization of product-form exchangeable feature probability functions

We characterize the class of exchangeable feature allocations assigning ...

0 Marco Battiston, et al. ∙

research

∙ 06/16/2016

The Mondrian Kernel

We introduce the Mondrian kernel, a fast random feature approximation to...

0 Matej Balog, et al. ∙

research

∙ 06/07/2016

Measuring the reliability of MCMC inference with bidirectional Monte Carlo

Markov chain Monte Carlo (MCMC) is one of the main workhorses of probabi...

0 Roger B. Grosse, et al. ∙

research

∙ 12/08/2015

Gibbs-type Indian buffet processes

We investigate a class of feature allocation models that generalize the ...

0 Creighton Heaukulani, et al. ∙

research

∙ 11/19/2015

Neural Network Matrix Factorization

Data often comes in the form of an array or matrix. Matrix factorization...

0 Gintare Karolina Dziugaite, et al. ∙

research

∙ 06/11/2015

Mondrian Forests for Large-Scale Regression when Uncertainty Matters

Many real-world regression problems demand a measure of the uncertainty ...

0 Balaji Lakshminarayanan, et al. ∙

research

∙ 05/14/2015

Training generative neural networks via Maximum Mean Discrepancy optimization

We consider training a deep neural network to generate samples from an u...

0 Gintare Karolina Dziugaite, et al. ∙

research

∙ 02/16/2015

Particle Gibbs for Bayesian Additive Regression Trees

Additive regression trees are flexible non-parametric models and popular...

0 Balaji Lakshminarayanan, et al. ∙

research

∙ 12/31/2014

The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof

We describe the combinatorial stochastic process underlying a sequence o...

0 Daniel M. Roy, et al. ∙

research

∙ 12/31/2013

The combinatorial structure of beta negative binomial processes

We characterize the combinatorial structure of conditionally-i.i.d. sequ...

0 Creighton Heaukulani, et al. ∙

research

∙ 12/30/2013

Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures

The natural habitat of most Bayesian methods is data represented by exch...

0 Peter Orbanz, et al. ∙

Daniel M. Roy

Featured Co-authors