research
∙
06/15/2023
Fit Like You Sample: Sample-Efficient Generalized Score Matching from Fast Mixing Markov Chains
Score matching is an approach to learning probability distributions para...
share
research
∙
06/03/2023
Provable benefits of score matching
Score matching is an alternative to maximum likelihood (ML) for estimati...
research
∙
06/01/2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation
Good data augmentation is one of the key factors that lead to the empiri...
research
∙
03/07/2023
How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding
While the successes of transformers across many domains are indisputable...
research
∙
10/21/2022
Neural Network Approximations of PDEs Beyond Linearity: Representational Perspective
A burgeoning line of research has developed deep neural networks capable...
research
∙
10/03/2022
Statistical Efficiency of Score Matching: The View from Isoperimetry
Deep generative models parametrized up to a normalizing constant (e.g. e...
research
∙
10/01/2022
Pitfalls of Gaussians as a noise distribution in NCE
Noise Contrastive Estimation (NCE) is a popular approach for learning pr...
research
∙
03/29/2022
Contrasting the landscape of contrastive and non-contrastive learning
A lot of recent advances in unsupervised feature learning are based on d...
research
∙
03/27/2022
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions
Continual learning is an emerging paradigm in machine learning, wherein ...
research
∙
02/18/2022
Masked prediction tasks: a parameter identifiability view
The vast majority of work in self-supervised learning, both theoretical ...
research
∙
02/17/2022
Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods
We consider Ising models on the hypercube with a general interaction mat...
research
∙
02/14/2022
Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization
A common explanation for the failure of deep networks to generalize out-...
research
∙
12/13/2021
Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias
Variational Autoencoders (VAEs) are one of the most commonly used genera...
research
∙
10/21/2021
Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation
Noise-contrastive estimation (NCE) is a statistically consistent method ...
research
∙
07/09/2021
The Effects of Invertibility on the Representational Complexity of Encoders in Variational Autoencoders
Training and using modern neural-network based latent-variable generativ...
research
∙
07/07/2021
Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flows
Normalizing flows are a widely used class of latent-variable generative ...
research
∙
06/18/2021
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments
Domain generalization aims at performing well on unseen test environment...
research
∙
06/03/2021
The Limitations of Limited Context for Constituency Parsing
Incorporating syntax into neural approaches in NLP has a multitude of pr...
research
∙
03/03/2021
Contrastive learning of strong-mixing continuous-time stochastic processes
Contrastive learning is a family of self-supervised methods where a mode...
research
∙
03/03/2021
Parametric Complexity Bounds for Approximating PDEs with Neural Networks
Recent empirical results show that deep networks can approximate solutio...
research
∙
02/25/2021
An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization
A popular assumption for out-of-distribution generalization is that the ...
research
∙
10/12/2020
The Risks of Invariant Risk Minimization
Invariant Causal Prediction (Peters et al., 2016) is a technique for out...
research
∙
10/02/2020
Representational aspects of depth and conditioning in normalizing flows
Normalizing flows are among the most popular paradigms in generative mod...
research
∙
09/30/2020
Efficient sampling from the Bingham distribution
We give a algorithm for exact sampling from the Bingham distribution p(x...
research
∙
08/11/2020
On Learning Language-Invariant Representations for Universal Machine Translation
The goal of universal machine translation is to learn to translate betwe...
research
∙
02/13/2020
Fast Convergence for Langevin Diffusion with Matrix Manifold Structure
In this paper, we study the problem of sampling from distributions of th...
research
∙
06/28/2019
Benefits of Overparameterization in Single-Layer Latent Variable Generative Models
One of the most surprising and exciting discoveries in supervising learn...
research
∙
05/30/2019
Sum-of-squares meets square loss: Fast rates for agnostic tensor completion
We study tensor completion in the agnostic setting. In the classical ten...
research
∙
11/29/2018
Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition
A key task in Bayesian machine learning is sampling from distributions t...
research
∙
08/22/2018
Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective
The free energy is a key quantity of interest in Ising models, but unfor...
research
∙
06/27/2018
Approximability of Discriminators Implies Diversity in GANs
While Generative Adversarial Networks (GANs) have empirically produced i...
research
∙
05/29/2018
Representational Power of ReLU Networks and Polynomial Kernels: Beyond Worst-Case Analysis
There has been a large amount of interest, both in the past and particul...
research
∙
11/07/2017
Theoretical limitations of Encoder-Decoder GAN architectures
Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an ...
research
∙
10/07/2017
Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo
A key task in Bayesian statistics is sampling from distributions that ar...
research
∙
06/14/2017
Provable benefits of representation learning
There is general consensus that learning representations is useful for a...
research
∙
04/29/2017
Extending and Improving Wordnet via Unsupervised Word Embeddings
This work presents an unsupervised approach for improving WordNet that b...
research
∙
12/28/2016
Provable learning of Noisy-or Networks
Many machine learning applications use latent variable models to explain...
research
∙
11/11/2016
Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates
Non-negative matrix factorization is a popular tool for decomposing data...
research
∙
07/12/2016
Approximate maximum entropy principles via Goemans-Williamson with applications to provable variational methods
The well known maximum-entropy principle due to Jaynes, which states tha...
research
∙
07/11/2016
How to calculate partition functions using convex programming hierarchies: provable bounds for variational methods
We consider the problem of approximating partition functions for Ising m...
research
∙
02/06/2016
Recovery guarantee of weighted low-rank approximation via alternating minimization
Many applications require recovering a ground truth low-rank matrix from...
research
∙
01/14/2016
Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Word embeddings are ubiquitous in NLP and information retrieval, but it'...
research
∙
03/23/2015
On some provably correct cases of variational inference for topic models
Variational inference is a very efficient and popular heuristic used in ...
research
∙
02/12/2015