Multi-task Learning and Catastrophic Forgetting in Continual Reinforcement Learning

by   João Ribeiro, et al.

In this paper we investigate two hypothesis regarding the use of deep reinforcement learning in multiple tasks. The first hypothesis is driven by the question of whether a deep reinforcement learning algorithm, trained on two similar tasks, is able to outperform two single-task, individually trained algorithms, by more efficiently learning a new, similar task, that none of the three algorithms has encountered before. The second hypothesis is driven by the question of whether the same multi-task deep RL algorithm, trained on two similar tasks and augmented with elastic weight consolidation (EWC), is able to retain similar performance on the new task, as a similar algorithm without EWC, whilst being able to overcome catastrophic forgetting in the two previous tasks. We show that a multi-task Asynchronous Advantage Actor-Critic (GA3C) algorithm, trained on Space Invaders and Demon Attack, is in fact able to outperform two single-tasks GA3C versions, trained individually for each single-task, when evaluated on a new, third task, namely, Phoenix. We also show that, when training two trained multi-task GA3C algorithms on the third task, if one is augmented with EWC, it is not only able to achieve similar performance on the new task, but also capable of overcoming a substantial amount of catastrophic forgetting on the two previous tasks.


Multi-task Deep Reinforcement Learning with PopArt

The reinforcement learning community has made great strides in designing...

Modular Networks Prevent Catastrophic Interference in Model-Based Multi-Task Reinforcement Learning

In a multi-task reinforcement learning setting, the learner commonly ben...

Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting

Neural networks can achieve extraordinary results on a wide variety of t...

Learning Vision-based Robotic Manipulation Tasks Sequentially in Offline Reinforcement Learning Settings

With the rise of deep reinforcement learning (RL) methods, many complex ...

Transferring Deep Reinforcement Learning with Adversarial Objective and Augmentation

In the past few years, deep reinforcement learning has been proven to so...

A Multi-Task Approach to Robust Deep Reinforcement Learning for Resource Allocation

With increasing complexity of modern communication systems, machine lear...

Domain adaptation strategies for cancer-independent detection of lymph node metastases

Recently, large, high-quality public datasets have led to the developmen...

Please sign up or login with your details

Forgot password? Click here to reset