Robustness-preserving Lifelong Learning via Dataset Condensation

by   Jinghan Jia, et al.

Lifelong learning (LL) aims to improve a predictive model as the data source evolves continuously. Most work in this learning paradigm has focused on resolving the problem of 'catastrophic forgetting,' which refers to a notorious dilemma between improving model accuracy over new data and retaining accuracy over previous data. Yet, it is also known that machine learning (ML) models can be vulnerable in the sense that tiny, adversarial input perturbations can deceive the models into producing erroneous predictions. This motivates the research objective of this paper - specification of a new LL framework that can salvage model robustness (against adversarial attacks) from catastrophic forgetting. Specifically, we propose a new memory-replay LL strategy that leverages modern bi-level optimization techniques to determine the 'coreset' of the current data (i.e., a small amount of data to be memorized) for ease of preserving adversarial robustness over time. We term the resulting LL framework 'Data-Efficient Robustness-Preserving LL' (DERPLL). The effectiveness of DERPLL is evaluated for class-incremental image classification using ResNet-18 over the CIFAR-10 dataset. Experimental results show that DERPLL outperforms the conventional coreset-guided LL baseline and achieves a substantial improvement in both standard accuracy and robust accuracy.


page 1

page 2

page 3

page 4


The Importance of Robust Features in Mitigating Catastrophic Forgetting

Continual learning (CL) is an approach to address catastrophic forgettin...

Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning

Class-incremental learning (CIL) suffers from the notorious dilemma betw...

Susceptibility of Continual Learning Against Adversarial Attacks

The recent advances in continual (incremental or lifelong) learning have...

AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning

Continual lifelong learning is an machine learning framework inspired by...

Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation

Continual relation extraction (CRE) aims to continually learn new relati...

Always Strengthen Your Strengths: A Drift-Aware Incremental Learning Framework for CTR Prediction

Click-through rate (CTR) prediction is of great importance in recommenda...

Please sign up or login with your details

Forgot password? Click here to reset