Generative Negative Replay for Continual Learning

by   Gabriele Graffieti, et al.
University of Bologna
University of Pisa

Learning continually is a key aspect of intelligence and a necessary ability to solve many real-life problems. One of the most effective strategies to control catastrophic forgetting, the Achilles' heel of continual learning, is storing part of the old data and replaying them interleaved with new experiences (also known as the replay approach). Generative replay, which is using generative models to provide replay patterns on demand, is particularly intriguing, however, it was shown to be effective mainly under simplified assumptions, such as simple scenarios and low-dimensional data. In this paper, we show that, while the generated data are usually not able to improve the classification accuracy for the old classes, they can be effective as negative examples (or antagonists) to better learn the new classes, especially when the learning experiences are small and contain examples of just one or few classes. The proposed approach is validated on complex class-incremental and data-incremental continual learning scenarios (CORe50 and ImageNet-1000) composed of high-dimensional data and a large number of training experiences: a setup where existing generative replay approaches usually fail.


page 1

page 2

page 3

page 4


Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

Humans learn all their life long. They accumulate knowledge from a seque...

Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition

We introduce a unified probabilistic approach for deep continual learnin...

Class-Incremental Learning with Repetition

Real-world data streams naturally include the repetition of previous con...

RECALL: Replay-based Continual Learning in Semantic Segmentation

Deep networks allow to obtain outstanding results in semantic segmentati...

Looking through the past: better knowledge retention for generative replay in continual learning

In this work, we improve the generative replay in a continual learning s...

Prediction Error-based Classification for Class-Incremental Learning

Class-incremental learning (CIL) is a particularly challenging variant o...

Architect, Regularize and Replay (ARR): a Flexible Hybrid Approach for Continual Learning

In recent years we have witnessed a renewed interest in machine learning...

Please sign up or login with your details

Forgot password? Click here to reset