Adaptation to criticality through organizational invariance in embodied agents

by   Miguel Aguilera, et al.

Many biological and cognitive systems do not operate deep within one or other regime of activity. Instead, they are poised at critical points located at transitions of their parameter space. The pervasiveness of criticality suggests that there may be general principles inducing this behaviour, yet there is no well-founded theory for understanding how criticality is found at a wide range of levels and contexts. In this paper we present a general adaptive mechanism that maintains an internal organizational structure in order to drive a system towards critical points while it interacts with different environments. We implement the mechanism in artificial embodied agents controlled by a neural network maintaining a correlation structure randomly sampled from an Ising model at critical temperature. Agents are evaluated in two classical reinforcement learning scenarios: the Mountain Car and the Acrobot double pendulum. In both cases the neural controller reaches a point of criticality, which coincides with a transition point between two regimes of the agent's behaviour. These results suggest that adaptation to criticality could be used as a general adaptive mechanism in some circumstances, providing an alternative explanation for the pervasive presence of criticality in biological and cognitive systems.


page 1

page 2

page 3

page 4


Criticality as It Could Be: organizational invariance as self-organized criticality in embodied agents

This paper outlines a methodological approach for designing adaptive age...

Learning Criticality in an Embodied Boltzmann Machine

Many biological and cognitive systems do not operate deep into one or ot...

Adaptive patch foraging in deep reinforcement learning agents

Patch foraging is one of the most heavily studied behavioral optimizatio...

Intelligent problem-solving as integrated hierarchical reinforcement learning

According to cognitive psychology and related disciplines, the developme...

Safe adaptation in multiagent competition

Achieving the capability of adapting to ever-changing environments is a ...

Bayesian Reinforcement Learning with Limited Cognitive Load

All biological and artificial agents must learn and make decisions given...

Please sign up or login with your details

Forgot password? Click here to reset