Don't forget, there is more than forgetting: new metrics for Continual Learning

by   Natalia Díaz Rodríguez, et al.
University of Bologna
ENSTA ParisTech

Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and forward knowledge transfer, memory overhead as well as computational efficiency. Drawing inspiration from the standard Multi-Attribute Value Theory (MAVT) we further propose to fuse these metrics into a single score for ranking purposes and we evaluate our proposal with five continual learning strategies on the iCIFAR-100 continual learning benchmark.


page 1

page 2

page 3

page 4


Gradient Episodic Memory for Continual Learning

One major obstacle towards AI is the poor ability of models to solve new...

Continual Learning for Robotics

Continual learning (CL) is a particular machine learning paradigm where ...

Continual Learning from Demonstration of Robotic Skills

Methods for teaching motion skills to robots focus on training for a sin...

New metrics for analyzing continual learners

Deep neural networks have shown remarkable performance when trained on i...

Learning to Learn: How to Continuously Teach Humans and Machines

Our education system comprises a series of curricula. For example, when ...

Fine-Grained Continual Learning

Robotic vision is a field where continual learning can play a significan...

A Comprehensive Survey of Continual Learning: Theory, Method and Application

To cope with real-world dynamics, an intelligent agent needs to incremen...

Please sign up or login with your details

Forgot password? Click here to reset