research
∙
07/24/2023
Nonparametric Linear Feature Learning in Regression Through Regularisation
Representation learning plays a crucial role in automated feature select...
share
research
∙
06/28/2023
Theory and applications of the Sum-Of-Squares technique
The Sum-of-Squares (SOS) approximation method is a technique used in opt...
research
∙
05/31/2023
Chain of Log-Concave Markov Chains
Markov chain Monte Carlo (MCMC) is a class of general-purpose algorithms...
research
∙
05/28/2023
On the impact of activation and normalization in obtaining isometric embeddings at initialization
In this paper, we explore the structure of the penultimate Gram matrix i...
research
∙
03/24/2023
The limited-memory recursive variational Gaussian approximation (L-RVGA)
We consider the problem of computing a Gaussian approximation to the pos...
research
∙
03/21/2023
Universal Smoothed Score Functions for Generative Modeling
We consider the problem of generative modeling based on smoothing an unk...
research
∙
03/16/2023
Variational Principles for Mirror Descent and Mirror Langevin Dynamics
Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a pr...
research
∙
03/06/2023
Convergence Rates for Non-Log-Concave Sampling and Log-Partition Estimation
Sampling from Gibbs distributions p(x) ∝exp(-V(x)/ε) and computing their...
research
∙
03/02/2023
High-dimensional analysis of double descent for linear regression with random projections
We consider linear regression problems with a varying number of random p...
research
∙
02/13/2023
Kernelized Diffusion maps
Spectral clustering and diffusion maps are celebrated dimensionality red...
research
∙
02/07/2023
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy
We present an algorithm for minimizing an objective with hard-to-compute...
research
∙
02/07/2023
On the relationship between multivariate splines and infinitely-wide neural networks
We consider multivariate splines and show that they have a random featur...
research
∙
11/10/2022
Regression as Classification: Influence of Task Formulation on Neural Network Features
Neural networks can be trained to solve regression problems by using gra...
research
∙
09/19/2022
On the Theoretical Properties of Noise Correlation in Stochastic Optimization
Studying the properties of stochastic noise to optimize complex non-conv...
research
∙
06/27/2022
Sum-of-Squares Relaxations for Information Theory and Variational Inference
We consider extensions of the Shannon relative entropy, referred to as f...
research
∙
06/15/2022
Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays
The existing analysis of asynchronous stochastic gradient descent (SGD) ...
research
∙
06/09/2022
Explicit Regularization in Overparametrized Models via Noise Injection
Injecting noise within gradient descent has several desirable features. ...
research
∙
05/31/2022
Variational inference via Wasserstein gradient flows
Along with Markov chain Monte Carlo (MCMC) methods, variational inferenc...
research
∙
05/26/2022
Active Labeling: Streaming Stochastic Gradients
The workhorse of machine learning is stochastic gradient descent. To acc...
research
∙
05/25/2022
Entropy Maximization with Depth: A Variational Principle for Random Neural Networks
To understand the essential role of depth in neural networks, we investi...
research
∙
05/25/2022
A systematic approach to Lyapunov analyses of continuous-time models in convex optimization
First-order methods are often analyzed via their continuous-time models,...
research
∙
05/25/2022
Fast Stochastic Composite Minimization and an Accelerated Frank-Wolfe Algorithm under Parallelization
We consider the problem of minimizing the sum of two convex functions. O...
research
∙
04/16/2022
Polynomial-time sparse measure recovery
How to recover a probability measure with sparse support from particular...
research
∙
04/11/2022
Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares
We consider potentially non-convex optimization problems, for which opti...
research
∙
02/17/2022
Information Theory with Kernel Methods
We consider the analysis of probability distributions through their asso...
research
∙
02/16/2022
On a Variance Reduction Correction of the Temporal Difference for Policy Evaluation in the Stochastic Continuous Setting
This paper deals with solving continuous time, state and action optimiza...
research
∙
02/06/2022
Anticorrelated Noise Injection for Improved Generalization
Injecting artificial noise into gradient descent (GD) is commonly employ...
research
∙
01/28/2022
Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics
We analyse the privacy leakage of noisy stochastic gradient descent by m...
research
∙
10/29/2021
Convergence of Uncertainty Sampling for Active Learning
Uncertainty sampling in active learning is heavily used in practice to r...
research
∙
10/20/2021
Sampling from Arbitrary Functions via PSD Models
In many areas of applied statistics and machine learning, generating an ...
research
∙
10/15/2021
Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization
Many supervised machine learning methods are naturally cast as optimizat...
research
∙
07/02/2021
Screening for a Reweighted Penalized Conditional Gradient Method
The conditional gradient method (CGM) is widely used in large-scale spar...
research
∙
06/10/2021
A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossip
We introduce the continuized Nesterov acceleration, a close variant of N...
research
∙
06/07/2021
Batch Normalization Orthogonalizes Representations in Deep Random Networks
This paper underlines a subtle property of batch-normalization (BN): Suc...
research
∙
05/31/2021
Max-Margin is Dead, Long Live Max-Margin!
The foundational concept of Max-Margin in machine learning is ill-posed ...
research
∙
02/11/2021
A Continuized View on Nesterov Acceleration
We introduce the "continuized" Nesterov acceleration, a close variant of...
research
∙
02/04/2021
Disambiguation of weak supervision with exponential convergence rates
Machine learning approached through supervised learning requires expensi...
research
∙
02/01/2021
Fast rates in structured prediction
Discrete supervised learning problems such as classification are often t...
research
∙
12/22/2020
Finding Global Minima via Kernel Approximations
We consider the global minimization of smooth functions based solely on ...
research
∙
10/02/2020
Variance-Reduced Methods for Machine Learning
Stochastic optimization lies at the heart of machine learning, and its c...
research
∙
09/30/2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes
Deep networks are often considered to be more expressive than shallow on...
research
∙
07/08/2020
Non-parametric Models for Non-negative Functions
Linear models have shown great effectiveness and flexibility in many fie...
research
∙
07/02/2020
Consistent Structured Prediction with Max-Min Margin Markov Networks
Max-margin methods for binary classification such as the support vector ...
research
∙
06/25/2020
Dual-Free Stochastic Decentralized Optimization with Variance Reduction
We consider the problem of training machine learning models on distribut...
research
∙
06/16/2020
Structured and Localized Image Restoration
We present a novel approach to image restoration that leverages ideas fr...
research
∙
06/15/2020
Tight Nonparametric Convergence Rates for Stochastic Gradient Descent under the Noiseless Linear Model
In the context of statistical supervised learning, the noiseless linear ...
research
∙
06/10/2020
Principled Analyses and Design of First-Order Methods with Inexact Proximal Operators
Proximal operations are among the most common primitives appearing in bo...
research
∙
06/08/2020
ARIANN: Low-Interaction Privacy-Preserving Deep Learning via Function Secret Sharing
We propose ARIANN, a low-interaction framework to perform private traini...
research
∙
05/20/2020
An Optimal Algorithm for Decentralized Finite Sum Optimization
Modern large-scale finite-sum optimization relies on two key aspects: di...
research
∙
03/30/2020