A good body is all you need: avoiding catastrophic interference via agent architecture search

by   Joshua Powers, et al.

In robotics, catastrophic interference continues to restrain policy training across environments. Efforts to combat catastrophic interference to date focus on novel neural architectures or training methods, with a recent emphasis on policies with good initial settings that facilitate training in new environments. However, none of these methods to date have taken into account how the physical architecture of the robot can obstruct or facilitate catastrophic interference, just as the choice of neural architecture can. In previous work we have shown how aspects of a robot's physical structure (specifically, sensor placement) can facilitate policy learning by increasing the fraction of optimal policies for a given physical structure. Here we show for the first time that this proxy measure of catastrophic interference correlates with sample efficiency across several search methods, proving that favorable loss landscapes can be induced by the correct choice of physical structure. We show that such structures can be found via co-optimization – optimization of a robot's structure and control policy simultaneously – yielding catastrophic interference resistant robot structures and policies, and that this is more efficient than control policy optimization alone. Finally, we show that such structures exhibit sensor homeostasis across environments and introduce this as the mechanism by which certain robots overcome catastrophic interference.


page 6

page 8


Embodiment dictates learnability in neural controllers

Catastrophic forgetting continues to severely restrict the learnability ...

Overcoming Catastrophic Interference by Conceptors

Catastrophic interference has been a major roadblock in the research of ...

On Catastrophic Interference in Atari 2600 Games

Model-free deep reinforcement learning algorithms are troubled with poor...

Neural Architecture Search of Deep Priors: Towards Continual Learning without Catastrophic Interference

In this paper we analyze the classification performance of neural networ...

Towards a practical measure of interference for reinforcement learning

Catastrophic interference is common in many network-based learning syste...

Measuring and Mitigating Interference in Reinforcement Learning

Catastrophic interference is common in many network-based learning syste...

Entropic Risk Measure in Policy Search

With the increasing pace of automation, modern robotic systems need to a...

Please sign up or login with your details

Forgot password? Click here to reset