Transformer with Peak Suppression and Knowledge Guidance for Fine-grained Image Recognition

by   Xinda Liu, et al.

Fine-grained image recognition is challenging because discriminative clues are usually fragmented, whether from a single image or multiple images. Despite their significant improvements, most existing methods still focus on the most discriminative parts from a single image, ignoring informative details in other regions and lacking consideration of clues from other associated images. In this paper, we analyze the difficulties of fine-grained image recognition from a new perspective and propose a transformer architecture with the peak suppression module and knowledge guidance module, which respects the diversification of discriminative features in a single image and the aggregation of discriminative clues among multiple images. Specifically, the peak suppression module first utilizes a linear projection to convert the input image into sequential tokens. It then blocks the token based on the attention response generated by the transformer encoder. This module penalizes the attention to the most discriminative parts in the feature learning process, therefore, enhancing the information exploitation of the neglected regions. The knowledge guidance module compares the image-based representation generated from the peak suppression module with the learnable knowledge embedding set to obtain the knowledge response coefficients. Afterwards, it formalizes the knowledge learning as a classification problem using response coefficients as the classification scores. Knowledge embeddings and image-based representations are updated during training so that the knowledge embedding includes discriminative clues for different images. Finally, we incorporate the acquired knowledge embeddings into the image-based representations as comprehensive representations, leading to significantly higher performance. Extensive evaluations on the six popular datasets demonstrate the advantage of the proposed method.


page 1

page 2

page 4

page 9

page 10


Complementary Frequency-Varying Awareness Network for Open-Set Fine-Grained Image Recognition

Open-set image recognition is a challenging topic in computer vision. Mo...

Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition

Learning subtle yet discriminative features (e.g., beak and eyes for a b...

ViT-FOD: A Vision Transformer based Fine-grained Object Discriminator

Recently, several Vision Transformer (ViT) based methods have been propo...

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

The core for tackling the fine-grained visual categorization (FGVC) is t...

Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition

Humans can naturally understand an image in depth with the aid of rich k...

Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples

Humans are capable of learning a new fine-grained concept with very litt...

Exploiting Fine-Grained DCT Representations for Hiding Image-Level Messages within JPEG Images

Unlike hiding bit-level messages, hiding image-level messages is more ch...

Please sign up or login with your details

Forgot password? Click here to reset