Neural Routing in Meta Learning

by   Jicang Cai, et al.

Meta-learning often referred to as learning-to-learn is a promising notion raised to mimic human learning by exploiting the knowledge of prior tasks but being able to adapt quickly to novel tasks. A plethora of models has emerged in this context and improved the learning efficiency, robustness, etc. The question that arises here is can we emulate other aspects of human learning and incorporate them into the existing meta learning algorithms? Inspired by the widely recognized finding in neuroscience that distinct parts of the brain are highly specialized for different types of tasks, we aim to improve the model performance of the current meta learning algorithms by selectively using only parts of the model conditioned on the input tasks. In this work, we describe an approach that investigates task-dependent dynamic neuron selection in deep convolutional neural networks (CNNs) by leveraging the scaling factor in the batch normalization (BN) layer associated with each convolutional layer. The problem is intriguing because the idea of helping different parts of the model to learn from different types of tasks may help us train better filters in CNNs, and improve the model generalization performance. We find that the proposed approach, neural routing in meta learning (NRML), outperforms one of the well-known existing meta learning baselines on few-shot classification tasks on the most widely used benchmark datasets.


page 1

page 2

page 3

page 4


Putting Theory to Work: From Learning Bounds to Meta-Learning Algorithms

Most of existing deep learning models rely on excessive amounts of label...

GaterNet: Dynamic Filter Selection in Convolutional Neural Network via a Dedicated Global Gating Network

The concept of conditional computation for deep nets has been proposed p...

Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?

Substantial efforts have been made on improving the generalization abili...

MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment

Recently, increasing interest has been drawn in exploiting deep convolut...

Learning to acquire novel cognitive tasks with evolution, plasticity and meta-meta-learning

In meta-learning, networks are trained with external algorithms to learn...

Learning-to-Learn Personalised Human Activity Recognition Models

Human Activity Recognition (HAR) is the classification of human movement...

Time Associated Meta Learning for Clinical Prediction

Rich Electronic Health Records (EHR), have created opportunities to impr...

Please sign up or login with your details

Forgot password? Click here to reset