Guido Montufar

research

∙ 05/31/2023

Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape

We study the loss landscape of two-layer mildly overparameterized ReLU n...

0 Kedar Karhadkar, et al. ∙

research

∙ 05/24/2023

Supermodular Rank: Set Function Decomposition and Optimization

We define the supermodular rank of a function on a lattice. This is the ...

0 Rishi Sonthalia, et al. ∙

research

∙ 04/12/2023

Function Space and Critical Points of Linear Convolutional Networks

We study the geometry of linear networks with one-dimensional convolutio...

0 Kathlén Kohn, et al. ∙

research

∙ 03/06/2023

Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss

We consider a deep matrix factorization model of covariance matrices tra...

0 Pierre Bréchet, et al. ∙

research

∙ 01/17/2023

Expected Gradients of Maxout Networks and Consequences to Parameter Initialization

We study the gradients of a maxout network with respect to inputs and pa...

0 Hanna Tseran, et al. ∙

research

∙ 11/15/2022

Characterizing the Spectrum of the NTK via a Power Series Expansion

Under mild conditions on the network initialization we derive a power se...

0 Michael Murray, et al. ∙

research

∙ 11/03/2022

Geometry and convergence of natural policy gradient methods

We study the convergence of several natural policy gradient (NPG) method...

0 Johannes Müller, et al. ∙

research

∙ 10/21/2022

FoSR: First-order spectral rewiring for addressing oversquashing in GNNs

Graph neural networks (GNNs) are able to leverage the structure of graph...

0 Kedar Karhadkar, et al. ∙

research

∙ 09/29/2022

Enumeration of max-pooling responses with generalized permutohedra

We investigate the combinatorics of max-pooling layers, which are functi...

0 Laura Escobar, et al. ∙

research

∙ 08/06/2022

Oversquashing in GNNs through the lens of information contraction and graph expansion

The quality of signal propagation in message-passing graph neural networ...

0 Pradeep Kr. Banerjee, et al. ∙

research

∙ 06/21/2022

On the effectiveness of persistent homology

Persistent homology (PH) is one of the most popular methods in Topologic...

0 Renata Turkeš, et al. ∙

research

∙ 06/06/2022

Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime

We provide quantitative bounds measuring the L^2 difference in function ...

0 Benjamin Bowman, et al. ∙

research

∙ 05/27/2022

Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space

Reward optimization in fully observable Markov decision processes is equ...

0 Johannes Müller, et al. ∙

research

∙ 04/23/2022

Continuity and Additivity Properties of Information Decompositions

Information decompositions quantify how the Shannon information about a ...

0 Johannes Rauh, et al. ∙

research

∙ 01/12/2022

Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks

We study the dynamics of a neural network in function space when optimiz...

0 Benjamin Bowman, et al. ∙

research

∙ 10/27/2021

Training Wasserstein GANs without gradient penalties

We propose a stable method to train Wasserstein generative adversarial n...

0 Dohyun Kwon, et al. ∙

research

∙ 10/23/2021

Learning curves for Gaussian process regression with power-law priors and targets

We study the power-law asymptotics of learning curves for Gaussian proce...

0 Hui Jin, et al. ∙

research

∙ 10/14/2021

The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs

We consider the problem of finding the best memoryless stochastic policy...

0 Johannes Müller, et al. ∙

research

∙ 08/03/2021

Geometry of Linear Convolutional Networks

We study the family of functions that are represented by a linear convol...

0 Kathlén Kohn, et al. ∙

research

∙ 07/01/2021

On the Expected Complexity of Maxout Networks

Learning with neural networks relies on the complexity of the representa...

3 Hanna Tseran, et al. ∙

research

∙ 06/23/2021

Weisfeiler and Lehman Go Cellular: CW Networks

Graph Neural Networks (GNNs) are limited in their expressive power, stru...

8 Cristian Bodnar, et al. ∙

research

∙ 05/04/2021

Information Complexity and Generalization Bounds

We present a unifying picture of PAC-Bayesian and mutual information-bas...

0 Pradeep Kr. Banerjee, et al. ∙

research

∙ 04/16/2021

Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums

We present results on the number of linear regions of the functions that...

0 Guido Montufar, et al. ∙

research

∙ 03/04/2021

Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks

The pairwise interaction paradigm of graph machine learning has predomin...

33 Cristian Bodnar, et al. ∙

research

∙ 12/21/2020

Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks

A recent line of work has analyzed the theoretical properties of deep ne...

0 Quynh Nguyen, et al. ∙

research

∙ 11/30/2020

Can neural networks learn persistent homology features?

Topological data analysis uses tools from topology – the mathematical ar...

0 Guido Montufar, et al. ∙

research

∙ 07/18/2020

Distributed Learning via Filtered Hyperinterpolation on Manifolds

Learning mappings of data on manifolds is an important topic in contempo...

8 Guido Montufar, et al. ∙

research

∙ 06/12/2020

Implicit bias of gradient descent for mean squared error regression with wide neural networks

We investigate gradient descent training of wide neural networks and the...

23 Hui Jin, et al. ∙

research

∙ 06/11/2020

Optimization Theory for ReLU Neural Networks Trained with Normalization Layers

The success of deep neural networks is in part due to the use of normali...

1 Yonatan Dukler, et al. ∙

research

∙ 03/15/2020

Wasserstein Distance to Independence Models

An independence model for discrete random variables is a Segre-Veronese ...

1 Turku Ozlum Celik, et al. ∙

research

∙ 10/22/2019

Stochastic Feedforward Neural Networks: Universal Approximation

In this chapter we take a look at the universal approximation question f...

21 Thomas Merkh, et al. ∙

research

∙ 10/21/2019

Kernelized Wasserstein Natural Gradient

Many machine learning problems can be expressed as the optimization of s...

13 Michael Arbel, et al. ∙

research

∙ 10/09/2019

How Well Do WGANs Estimate the Wasserstein Metric?

Generative modelling is often cast as minimizing a similarity measure be...

16 Anton Mallasto, et al. ∙

research

∙ 09/25/2019

HaarPooling: Graph Pooling with Compressive Haar Basis

Deep Graph Neural Networks (GNNs) are instrumental in graph classificati...

14 Yu Guang Wang, et al. ∙

research

∙ 09/15/2019

Wasserstein Diffusion Tikhonov Regularization

We propose regularization strategies for learning discriminative models ...

12 Alex Tong Lin, et al. ∙

research

∙ 06/13/2019

Factorized Mutual Information Maximization

We investigate the sets of joint probability distributions that maximize...

0 Thomas Merkh, et al. ∙

research

∙ 10/27/2018

The Variational Deficiency Bottleneck

We introduce a bottleneck method for learning data representations based...

0 Pradeep Kr. Banerjee, et al. ∙

research

∙ 07/18/2018

Ricci curvature for parametric statistics via optimal transport

We elaborate the notion of a Ricci curvature lower bound for parametrize...

1 Wuchen Li, et al. ∙

research

∙ 06/19/2018

Restricted Boltzmann Machines: Introduction and Review

The restricted Boltzmann machine is a network of stochastic units with u...

0 Guido Montufar, et al. ∙

research

∙ 03/16/2018

Natural gradient via optimal transport I

We study a natural Wasserstein gradient flow on manifolds of probability...

1 Wuchen Li, et al. ∙

research

∙ 09/15/2017

Mixtures and products in two graphical models

We compare two statistical models of three binary random variables. One ...

1 Anna Seigal, et al. ∙

research

∙ 04/06/2017

Geometry of Policy Improvement

We investigate the geometry of optimal memoryless time independent decis...

0 Guido Montufar, et al. ∙

research

∙ 05/31/2016

Information Theoretically Aided Reinforcement Learning for Embodied Agents

Reinforcement learning for embodied agents is a challenging problem. The...

0 Guido Montufar, et al. ∙

research

∙ 12/01/2015

Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Human Hopping

In the context of embodied artificial intelligence, morphological comput...

0 Keyan Ghazi-Zahedi, et al. ∙

research

∙ 11/10/2015

Dimension of Marginals of Kronecker Product Models

A Kronecker product model is the set of visible marginal probability dis...

0 Guido Montufar, et al. ∙

research

∙ 08/14/2015

Hierarchical Models as Marginals of Hierarchical Models

We investigate the representation of hierarchical models in terms of mar...

0 Guido Montufar, et al. ∙

research

∙ 03/24/2015

Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks

We establish upper bounds for the minimal number of hidden units for whi...

0 Guido Montufar, et al. ∙

research

∙ 03/24/2015

Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes

It is well known that for any finite state Markov decision process (MDP)...

0 Guido Montufar, et al. ∙

research

∙ 11/14/2014

Deep Narrow Boltzmann Machines are Universal Approximators

We show that deep narrow Boltzmann machines are universal approximators ...

0 Guido Montufar, et al. ∙

research

∙ 02/14/2014

Geometry and Expressive Power of Conditional Restricted Boltzmann Machines

Conditional restricted Boltzmann machines are undirected stochastic neur...

0 Guido Montufar, et al. ∙

Guido Montufar

Featured Co-authors