Ex-Model: Continual Learning from a Stream of Trained Models

12/13/2021
by   Antonio Carta, et al.
Scuola Normale Superiore
University of Pisa
10

Learning continually from non-stationary data streams is a challenging research topic of growing popularity in the last few years. Being able to learn, adapt, and generalize continually in an efficient, effective, and scalable way is fundamental for a sustainable development of Artificial Intelligent systems. However, an agent-centric view of continual learning requires learning directly from raw data, which limits the interaction between independent agents, the efficiency, and the privacy of current approaches. Instead, we argue that continual learning systems should exploit the availability of compressed information in the form of trained models. In this paper, we introduce and formalize a new paradigm named "Ex-Model Continual Learning" (ExML), where an agent learns from a sequence of previously trained models instead of raw data. We further contribute with three ex-model continual learning algorithms and an empirical setting comprising three datasets (MNIST, CIFAR-10 and CORe50), and eight scenarios, where the proposed algorithms are extensively tested. Finally, we highlight the peculiarities of the ex-model paradigm and we point out interesting future research directions.

READ FULL TEXT

page 7

page 13

page 14

page 15

page 16

10/01/2020

Meta-Consolidation for Continual Learning

The ability to continuously learn and adapt itself to new tasks, without...
04/15/2021

Rehearsal revealed: The limits and merits of revisiting samples in continual learning

Learning from non-stationary data streams and overcoming catastrophic fo...
12/09/2022

PIVOT: Prompting for Video Continual Learning

Modern machine learning pipelines are limited due to data availability, ...
05/26/2021

Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks

Continual learning is essential for all real-world applications, as froz...
08/02/2021

Sequoia: A Software Framework to Unify Continual Learning Research

The field of Continual Learning (CL) seeks to develop algorithms that ac...
10/30/2018

Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines

Continual learning has received a great deal of attention recently with ...

Please sign up or login with your details

Forgot password? Click here to reset