Meta Networks

by   Tsendsuren Munkhdalai, et al.

Neural networks have been successfully applied in applications with a large amount of labeled data. However, the task of rapid generalization on new concepts with small training data while preserving performances on previously learned ones still presents a significant challenge to neural network models. In this work, we introduce a novel meta learning method, Meta Networks (MetaNet), that learns a meta-level knowledge across tasks and shifts its inductive biases via fast parameterization for rapid generalization. When evaluated on Omniglot and Mini-ImageNet benchmarks, our MetaNet models achieve a near human-level performance and outperform the baseline approaches by up to 6 generalization and continual learning.


page 1

page 2

page 3

page 4


Meta-Consolidation for Continual Learning

The ability to continuously learn and adapt itself to new tasks, without...

Learning where to learn: Gradient sparsity in meta and continual learning

Finding neural network weights that generalize well from small datasets ...

Few-Shot Unsupervised Continual Learning through Meta-Examples

In real-world applications, data do not reflect the ones commonly used f...

Learning to Continually Learn Rapidly from Few and Noisy Data

Neural networks suffer from catastrophic forgetting and are unable to se...

Generalising via Meta-Examples for Continual Learning in the Wild

Learning quickly and continually is still an ambitious task for neural n...

Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks

In this work, we propose a novel prior learning method for advancing gen...

Generalization in Neural Networks: A Broad Survey

This paper reviews concepts, modeling approaches, and recent findings al...

Please sign up or login with your details

Forgot password? Click here to reset