research
∙
09/07/2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck
This work investigates the nuanced algorithm design choices for deep lea...
share
research
∙
06/22/2023
Adversarial Resilience in Sequential Prediction via Abstention
We study the problem of sequential prediction in the stochastic setting ...
research
∙
06/01/2023
Exposing Attention Glitches with Flip-Flop Language Modeling
Why do large language models sometimes output factual inaccuracies and e...
research
∙
04/20/2023
Learning Narrow One-Hidden-Layer ReLU Networks
We consider the well-studied problem of learning a linear combination of...
research
∙
10/19/2022
Transformers Learn Shortcuts to Automata
Algorithmic reasoning requires capabilities which are most naturally und...
research
∙
09/01/2022
Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms
Neural Networks (NNs) struggle to efficiently learn certain problems, su...
research
∙
07/18/2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit
There is mounting empirical evidence of emergent phenomena in the capabi...
research
∙
02/28/2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases
Contrastive learning is a popular form of self-supervised learning that ...
research
∙
10/21/2021
Anti-Concentrated Confidence Bonuses for Scalable Exploration
Intrinsic rewards play a central role in handling the exploration-exploi...
research
∙
10/19/2021
Inductive Biases and Variable Creation in Self-Attention Mechanisms
Self-attention, an architectural motif designed to model long-range inte...
research
∙
07/20/2021
Statistical Estimation from Dependent Data
We consider a general statistical estimation problem wherein binary labe...
research
∙
06/18/2021
Investigating the Role of Negatives in Contrastive Representation Learning
Noise contrastive learning is a popular technique for unsupervised repre...
research
∙
06/17/2021
Gone Fishing: Neural Active Learning with Fisher Embeddings
There is an increasing need for effective active learning algorithms tha...
research
∙
03/01/2021
Acceleration via Fractal Learning Rate Schedules
When balancing the practical tradeoffs of iterative methods for large-sc...
research
∙
11/27/2020
Tight Hardness Results for Training Depth-2 ReLU Networks
We prove several hardness results for training depth-2 neural networks w...
research
∙
07/25/2020
From Boltzmann Machines to Neural Networks and Back Again
Graphical models are powerful tools for modeling high-dimensional data, ...
research
∙
06/29/2020
Statistical-Query Lower Bounds via Functional Gradients
We give the first statistical-query lower bounds for agnostically learni...
research
∙
06/22/2020
Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent
We prove the first superpolynomial lower bounds for learning one-layer n...
research
∙
05/26/2020
Approximation Schemes for ReLU Regression
We consider the fundamental problem of ReLU regression, where the goal i...
research
∙
05/15/2020
Efficiently Learning Adversarially Robust Halfspaces with Noise
We study the problem of learning adversarially robust halfspaces in the ...
research
∙
11/04/2019
Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals
We consider the problem of computing the best-fitting ReLU with respect ...
research
∙
06/15/2019
Learning Restricted Boltzmann Machines with Arbitrary External Fields
We study the problem of learning graphical models with latent variables....
research
∙
06/14/2019
Disentangling Mixtures of Epidemics on Graphs
We consider the problem of learning the weighted edges of a mixture of t...
research
∙
03/21/2019
Learning Two layer Networks with Multinomial Activation and High Thresholds
Giving provable guarantees for learning neural networks is a core challe...
research
∙
02/21/2019
Quantifying Perceptual Distortion of Adversarial Examples
Recent work has shown that additive threat models, which only permit the...
research
∙
02/13/2019
Learning Ising Models with Independent Failures
We give the first efficient algorithm for learning the structure of an I...
research
∙
05/20/2018
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps
We propose a new algorithm to learn a one-hidden-layer convolutional neu...
research
∙
02/07/2018
Learning One Convolutional Layer with Overlapping Patches
We give the first provably efficient algorithm for learning a one hidden...
research
∙
09/18/2017