A Unified Meta-Learning Framework for Dynamic Transfer Learning

by   Jun Wu, et al.

Transfer learning refers to the transfer of knowledge or information from a relevant source task to a target task. However, most existing works assume both tasks are sampled from a stationary task distribution, thereby leading to the sub-optimal performance for dynamic tasks drawn from a non-stationary task distribution in real scenarios. To bridge this gap, in this paper, we study a more realistic and challenging transfer learning setting with dynamic tasks, i.e., source and target tasks are continuously evolving over time. We theoretically show that the expected error on the dynamic target task can be tightly bounded in terms of source knowledge and consecutive distribution discrepancy across tasks. This result motivates us to propose a generic meta-learning framework L2E for modeling the knowledge transferability on dynamic tasks. It is centered around a task-guided meta-learning problem with a group of meta-pairs of tasks, based on which we are able to learn the prior model initialization for fast adaptation on the newest target task. L2E enjoys the following properties: (1) effective knowledge transferability across dynamic tasks; (2) fast adaptation to the new target task; (3) mitigation of catastrophic forgetting on historical target tasks; and (4) flexibility in incorporating any existing static transfer learning algorithms. Extensive experiments on various image data sets demonstrate the effectiveness of the proposed L2E framework.


page 1

page 2

page 3

page 4


Non-IID Transfer Learning on Graphs

Transfer learning refers to the transfer of knowledge or information fro...

Dynamic Transfer Learning across Graphs

Transferring knowledge across graphs plays a pivotal role in many high-s...

Robustifying Sequential Neural Processes

When tasks change over time, meta-transfer learning seeks to improve the...

Transferring Knowledge across Learning Processes

In complex transfer learning scenarios new tasks might not be tightly li...

Transfer Learning with Kernel Methods

Transfer learning refers to the process of adapting a model trained on a...

Continuous Transfer Learning with Label-informed Distribution Alignment

Transfer learning has been successfully applied across many high-impact ...

Predicting Water Temperature Dynamics of Unmonitored Lakes with Meta Transfer Learning

Most environmental data come from a minority of well-observed sites. An ...

Please sign up or login with your details

Forgot password? Click here to reset