Behind the Machine's Gaze: Biologically Constrained Neural Networks Exhibit Human-like Visual Attention

by   Leo Schwinn, et al.

By and large, existing computational models of visual attention tacitly assume perfect vision and full access to the stimulus and thereby deviate from foveated biological vision. Moreover, modelling top-down attention is generally reduced to the integration of semantic features without incorporating the signal of a high-level visual tasks that have shown to partially guide human attention. We propose the Neural Visual Attention (NeVA) algorithm to generate visual scanpaths in a top-down manner. With our method, we explore the ability of neural networks on which we impose the biological constraints of foveated vision to generate human-like scanpaths. Thereby, the scanpaths are generated to maximize the performance with respect to the underlying visual task (i.e., classification or reconstruction). Extensive experiments show that the proposed method outperforms state-of-the-art unsupervised human attention models in terms of similarity to human scanpaths. Additionally, the flexibility of the framework allows to quantitatively investigate the role of different tasks in the generated visual behaviours. Finally, we demonstrate the superiority of the approach in a novel experiment that investigates the utility of scanpaths in real-world applications, where imperfect viewing conditions are given.


page 3

page 15


Simulating Human Gaze with Neural Visual Attention

Existing models of human visual attention are generally unable to incorp...

Peripheral Vision Transformer

Human vision possesses a special type of visual processing systems calle...

Top-Down Visual Attention from Analysis by Synthesis

Current attention algorithms (e.g., self-attention) are stimulus-driven ...

Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements

Deep Convolutional Neural Networks (DCNNs) were originally inspired by p...

Contrastive Language-Image Pretrained Models are Zero-Shot Human Scanpath Predictors

Understanding the mechanisms underlying human attention is a fundamental...

A Biologically Inspired Visual Working Memory for Deep Networks

The ability to look multiple times through a series of pose-adjusted gli...

Please sign up or login with your details

Forgot password? Click here to reset