Deconstructing deep active inference

by   Théophile Champion, et al.

Active inference is a theory of perception, learning and decision making, which can be applied to neuroscience, robotics, and machine learning. Recently, reasearch has been taking place to scale up this framework using Monte-Carlo tree search and deep learning. The goal of this activity is to solve more complicated tasks using deep active inference. First, we review the existing literature, then, we progresively build a deep active inference agent. For two agents, we have experimented with five definitions of the expected free energy and three different action selection strategies. According to our experiments, the models able to solve the dSprites environment are the ones that maximise rewards. Finally, we compare the similarity of the representation learned by the layers of various agents using centered kernel alignment. Importantly, the agent maximising reward and the agent minimising expected free energy learn very similar representations except for the last layer of the critic network (reflecting the difference in learning objective), and the variance layers of the transition and encoder networks. We found that the reward maximising agent is a lot more certain than the agent minimising expected free energy. This is because the agent minimising expected free energy always picks the action down, and does not gather enough data for the other actions. In contrast, the agent maximising reward, keeps on selecting the actions left and right, enabling it to successfully solve the task. The only difference between those two agents is the epistemic value, which aims to make the outputs of the transition and encoder networks as close as possible. Thus, the agent minimising expected free energy picks a single action (down), and becomes an expert at predicting the future when selecting this action. This makes the KL divergence between the output of the transition and encoder networks small.


page 7

page 21

page 33

page 35

page 39

page 40

page 41

page 42


Deep active inference agents using Monte-Carlo methods

Active inference is a Bayesian framework for understanding biological in...

A Neural Active Inference Model of Perceptual-Motor Learning

The active inference framework (AIF) is a promising new computational fr...

Whence the Expected Free Energy?

The Expected Free Energy (EFE) is a central quantity in the theory of ac...

A Layered Architecture for Active Perception: Image Classification using Deep Reinforcement Learning

We propose a planning and perception mechanism for a robot (agent), that...

Successor Representation Active Inference

Recent work has uncovered close links between between classical reinforc...

Contrastive Active Inference

Active inference is a unifying theory for perception and action resting ...

Towards neoRL networks; the emergence of purposive graphs

The neoRL framework for purposive AI implements latent learning by emula...

Please sign up or login with your details

Forgot password? Click here to reset