Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning

by   Sam Blakeman, et al.

The human brain uses selective attention to filter perceptual input so that only the components that are useful for behaviour are processed using its limited computational resources. We focus on one particular form of visual attention known as feature-based attention, which is concerned with identifying features of the visual input that are important for the current task regardless of their spatial location. Visual feature-based attention has been proposed to improve the efficiency of Reinforcement Learning (RL) by reducing the dimensionality of state representations and guiding learning towards relevant features. Despite achieving human level performance in complex perceptual-motor tasks, Deep RL algorithms have been consistently criticised for their poor efficiency and lack of flexibility. Visual feature-based attention therefore represents one option for addressing these criticisms. Nevertheless, it is still an open question how the brain is able to learn which features to attend to during RL. To help answer this question we propose a novel algorithm, termed Selective Particle Attention (SPA), which imbues a Deep RL agent with the ability to perform selective feature-based attention. SPA learns which combinations of features to attend to based on their bottom-up saliency and how accurately they predict future reward. We evaluate SPA on a multiple choice task and a 2D video game that both involve raw pixel input and dynamic changes to the task structure. We show various benefits of SPA over approaches that naively attend to either all or random subsets of features. Our results demonstrate (1) how visual feature-based attention in Deep RL models can improve their learning efficiency and ability to deal with sudden changes in task structure and (2) that particle filters may represent a viable computational account of how visual feature-based attention occurs in the brain.


page 8

page 11

page 12

page 13


An Initial Attempt of Combining Visual Selective Attention with Deep Reinforcement Learning

Visual attention serves as a means of feature selection mechanism in the...

Learning to predict where to look in interactive environments using deep recurrent q-learning

Bottom-Up (BU) saliency models do not perform well in complex interactiv...

A Complementary Learning Systems Approach to Temporal Difference Learning

Complementary Learning Systems (CLS) theory suggests that the brain uses...

Human versus Machine Attention in Deep Reinforcement Learning Tasks

Deep reinforcement learning (RL) algorithms are powerful tools for solvi...

Learning to attend in a brain-inspired deep neural network

Recent machine learning models have shown that including attention as a ...

Speeding up reinforcement learning by combining attention and agency features

When playing video-games we immediately detect which entity we control a...

Please sign up or login with your details

Forgot password? Click here to reset