Class-Incremental Learning with Repetition

01/26/2023
by   Hamed Hemati, et al.
University of Bologna
Scuola Normale Superiore
University of Pisa
University of St. Gallen
13

Real-world data streams naturally include the repetition of previous concepts. From a Continual Learning (CL) perspective, repetition is a property of the environment and, unlike replay, cannot be controlled by the user. Nowadays, Class-Incremental scenarios represent the leading test-bed for assessing and comparing CL strategies. This family of scenarios is very easy to use, but it never allows revisiting previously seen classes, thus completely disregarding the role of repetition. We focus on the family of Class-Incremental with Repetition (CIR) scenarios, where repetition is embedded in the definition of the stream. We propose two stochastic scenario generators that produce a wide range of CIR scenarios starting from a single dataset and a few control parameters. We conduct the first comprehensive evaluation of repetition in CL by studying the behavior of existing CL strategies under different CIR scenarios. We then present a novel replay strategy that exploits repetition and counteracts the natural imbalance present in the stream. On both CIFAR100 and TinyImageNet, our strategy outperforms other replay approaches, which are not designed for environments with repetition.

READ FULL TEXT

page 8

page 17

12/06/2021

Is Class-Incremental Enough for Continual Learning?

The ability of a model to learn continually can be empirically assessed ...
04/12/2022

Generative Negative Replay for Continual Learning

Learning continually is a key aspect of intelligence and a necessary abi...
05/28/2023

Just a Glimpse: Rethinking Temporal Information for Video Continual Learning

Class-incremental learning is one of the most important settings for the...
01/06/2023

Architect, Regularize and Replay (ARR): a Flexible Hybrid Approach for Continual Learning

In recent years we have witnessed a renewed interest in machine learning...
05/29/2023

SHARP: Sparsity and Hidden Activation RePlay for Neuro-Inspired Continual Learning

Deep neural networks (DNNs) struggle to learn in dynamic environments si...
03/20/2020

Online Continual Learning on Sequences

Online continual learning (OCL) refers to the ability of a system to lea...
07/11/2023

EgoAdapt: A multi-stream evaluation study of adaptation to real-world egocentric user video

In egocentric action recognition a single population model is typically ...

Please sign up or login with your details

Forgot password? Click here to reset