Information Theoretic Meta Learning with Gaussian Processes

by   Michalis K. Titsias, et al.

We formulate meta learning using information theoretic concepts such as mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training or support set, that is highly informative about predicting the validation set. By making use of variational approximations to the mutual information we derive a general and tractable framework for meta learning. We particularly develop new memory-based meta learning algorithms based on Gaussian processes and derive extensions that combine memory and gradient based meta learning. We demonstrate our method on few-shot regression and classification by using standard benchmarks such as Omniglot, mini-Imagenet and Augmented Omniglot.


page 1

page 2

page 3

page 4


Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis

We derive a novel information-theoretic analysis of the generalization p...

Discrete Infomax Codes for Meta-Learning

Learning compact discrete representations of data is itself a key task i...

Bayesian Meta-Learning Through Variational Gaussian Processes

Recent advances in the field of meta-learning have tackled domains consi...

Information-Theoretic Analysis of Epistemic Uncertainty in Bayesian Meta-learning

The overall predictive uncertainty of a trained predictor can be decompo...

Conditional Mutual Information Bound for Meta Generalization Gap

Meta-learning infers an inductive bias—typically in the form of the hype...

An Information-theoretic Progressive Framework for Interpretation

Both brain science and the deep learning communities have the problem of...

Meta-Learning via Feature-Label Memory Network

Deep learning typically requires training a very capable architecture us...

Please sign up or login with your details

Forgot password? Click here to reset