research
∙
09/15/2023
Scaling Laws for Sparsely-Connected Foundation Models
We explore the impact of parameter sparsity on the scaling behavior of T...
share
research
∙
08/02/2023
From Sparse to Soft Mixtures of Experts
Sparse mixture of expert architectures (MoEs) scale model capacity witho...
research
∙
10/19/2022
On the Adversarial Robustness of Mixture of Experts
Adversarial robustness is a key desirable property of neural networks. I...
research
∙
09/14/2022
PaLI: A Jointly-Scaled Multilingual Language-Image Model
Effective scaling and a flexible task interface enable large language mo...
research
∙
06/06/2022
Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
Large sparsely-activated models have obtained excellent performance in m...
research
∙
02/24/2022
Learning to Merge Tokens in Vision Transformers
Transformers are widely applied to solve natural language understanding ...
research
∙
06/10/2021
Scaling Vision with Sparse Mixture of Experts
Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated exce...
research
∙
10/14/2020
Deep Ensembles for Low-Data Transfer Learning
In the low-data regime, it is difficult to train good supervised models ...
research
∙
10/13/2020
Which Model to Transfer? Finding the Needle in the Growing Haystack
Transfer learning has been recently popularized as a data-efficient alte...
research
∙
09/28/2020
Scalable Transfer Learning with Expert Models
Transfer of pre-trained representations can improve sample efficiency an...
research
∙
01/22/2020
On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
Uncertainty quantification for deep learning is a challenging open probl...
research
∙
07/25/2019
Google Research Football: A Novel Reinforcement Learning Environment
Recent progress in the field of reinforcement learning has been accelera...
research
∙
06/19/2019
Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates
We consider the core reinforcement-learning problem of on-policy value f...
research
∙
05/27/2019
Practical and Consistent Estimation of f-Divergences
The estimation of an f-divergence between two probability distributions ...
research
∙
02/26/2018
Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling
Recent advances in deep reinforcement learning have made significant str...
research
∙
03/02/2017
Active Learning for Accurate Estimation of Linear Models
We explore the sequential decision making problem where the goal is to e...
research
∙
03/01/2017
Human Interaction with Recommendation Systems: On Bias and Exploration
Recommendation systems rely on historical user data to provide suggestio...
research
∙
02/09/2016