A Comparative Study of Calibration Methods for Imbalanced Class Incremental Learning

by   Umang Aggarwal, et al.

Deep learning approaches are successful in a wide range of AI problems and in particular for visual recognition tasks. However, there are still open problems among which is the capacity to handle streams of visual information and the management of class imbalance in datasets. Existing research approaches these two problems separately while they co-occur in real world applications. Here, we study the problem of learning incrementally from imbalanced datasets. We focus on algorithms which have a constant deep model complexity and use a bounded memory to store exemplars of old classes across incremental states. Since memory is bounded, old classes are learned with fewer images than new classes and an imbalance due to incremental learning is added to the initial dataset imbalance. A score prediction bias in favor of new classes appears and we evaluate a comprehensive set of score calibration methods to reduce it. Evaluation is carried with three datasets, using two dataset imbalance configurations and three bounded memory sizes. Results show that most calibration methods have beneficial effect and that they are most useful for lower bounded memory sizes, which are most interesting in practice. As a secondary contribution, we remove the usual distillation component from the loss function of incremental learning algorithms. We show that simpler vanilla fine tuning is a stronger backbone for imbalanced incremental learning algorithms.


page 1

page 2

page 3

page 4


ScaIL: Classifier Weights Scaling for Class Incremental Learning

Incremental learning is useful if an AI agent needs to integrate data fr...

Active Class Incremental Learning for Imbalanced Datasets

Incremental Learning (IL) allows AI systems to adapt to streamed data. M...

Initial Classifier Weights Replay for Memoryless Class Incremental Learning

Incremental Learning (IL) is useful when artificial systems need to deal...

Drift-Aware Multi-Memory Model for Imbalanced Data Streams

Online class imbalance learning deals with data streams that are affecte...

Multi-Granularity Regularized Re-Balancing for Class Incremental Learning

Deep learning models suffer from catastrophic forgetting when learning n...

How Sensitive are Meta-Learners to Dataset Imbalance?

Meta-Learning (ML) has proven to be a useful tool for training Few-Shot ...

Sequential Targeting: an incremental learning approach for data imbalance in text classification

Classification tasks require a balanced distribution of data to ensure t...

Please sign up or login with your details

Forgot password? Click here to reset