DeepAI AI Chat
Log In Sign Up

Conditionally Learn to Pay Attention for Sequential Visual Task

11/11/2019
by   Jun He, et al.
Nanjing Normal University
Nanjing University of Information Science and Technology
NetEase, Inc
58

Sequential visual task usually requires to pay attention to its current interested object conditional on its previous observations. Different from popular soft attention mechanism, we propose a new attention framework by introducing a novel conditional global feature which represents the weak feature descriptor of the current focused object. Specifically, for a standard CNN (Convolutional Neural Network) pipeline, the convolutional layers with different receptive fields are used to produce the attention maps by measuring how the convolutional features align to the conditional global feature. The conditional global feature can be generated by different recurrent structure according to different visual tasks, such as a simple recurrent neural network for multiple objects recognition, or a moderate complex language model for image caption. Experiments show that our proposed conditional attention model achieves the best performance on the SVHN (Street View House Numbers) dataset with / without extra bounding box; and for image caption, our attention model generates better scores than the popular soft attention model.

READ FULL TEXT

page 2

page 4

page 6

page 9

page 10

page 12

page 17

page 18

12/11/2018

Reproduction Report on "Learn to Pay Attention"

We have successfully implemented the "Learn to Pay Attention" model of a...
03/09/2016

Recursive Recurrent Nets with Attention Modeling for OCR in the Wild

We present recursive recurrent neural networks with attention modeling (...
01/16/2019

UAN: Unified Attention Network for Convolutional Neural Networks

We propose a new architecture that learns to attend to different Convolu...
05/04/2017

Recurrent Soft Attention Model for Common Object Recognition

We propose the Recurrent Soft Attention Model, which integrates the visu...
04/06/2018

Learn To Pay Attention

We propose an end-to-end-trainable attention module for convolutional ne...
05/12/2016

Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model

Recent advances in conditional recurrent language modelling have mainly ...
09/26/2018

Pay attention! - Robustifying a Deep Visuomotor Policy through Task-Focused Attention

Several recent projects demonstrated the promise of end-to-end learned d...

Code Repositories

ConditionalLearnToPayAttention

None


view repo