Where to Focus: Deep Attention-based Spatially Recurrent Bilinear Networks for Fine-Grained Visual Recognition

by   Lin Wu, et al.

Fine-grained visual recognition typically depends on modeling subtle difference from object parts. However, these parts often exhibit dramatic visual variations such as occlusions, viewpoints, and spatial transformations, making it hard to detect. In this paper, we present a novel attention-based model to automatically, selectively and accurately focus on critical object regions with higher importance against appearance variations. Given an image, two different Convolutional Neural Networks (CNNs) are constructed, where the outputs of two CNNs are correlated through bilinear pooling to simultaneously focus on discriminative regions and extract relevant features. To capture spatial distributions among the local regions with visual attention, soft attention based spatial Long-Short Term Memory units (LSTMs) are incorporated to realize spatially recurrent yet visually selective over local input patterns. All the above intuitions equip our network with the following novel model: two-stream CNN layers, bilinear pooling layer, spatial recurrent layer with location attention are jointly trained via an end-to-end fashion to serve as the part detector and feature extractor, whereby relevant features are localized and extracted attentively. We show the significance of our network against two well-known visual recognition tasks: fine-grained image classification and person re-identification.


page 1

page 2

page 3

page 4


Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition

Fine-grained visual recognition is challenging because it highly relies ...

Bilinear CNNs for Fine-grained Visual Recognition

We present a simple and effective architecture for fine-grained visual r...

Where to Look and How to Describe: Fashion Image Retrieval with an Attentional Heterogeneous Bilinear Network

Fashion products typically feature in compositions of a variety of style...

Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-Boosting Attention Mechanism

The challenge of fine-grained visual recognition often lies in discoveri...

Improve the Interpretability of Attention: A Fast, Accurate, and Interpretable High-Resolution Attention Model

The prevalence of employing attention mechanisms has brought along conce...

Multiregion Bilinear Convolutional Neural Networks for Person Re-Identification

In this work we propose a new architecture for person re-identification....

DAF-NET: a saliency based weakly supervised method of dual attention fusion for fine-grained image classification

Fine-grained image classification is a challenging problem, since the di...

Please sign up or login with your details

Forgot password? Click here to reset