Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems

by   Yangyang Shu, et al.

Self-supervised learning (SSL) strategies have demonstrated remarkable performance in various recognition tasks. However, both our preliminary investigation and recent studies suggest that they may be less effective in learning representations for fine-grained visual recognition (FGVR) since many features helpful for optimizing SSL objectives are not suitable for characterizing the subtle differences in FGVR. To overcome this issue, we propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes, dubbed as common rationales in this paper. Intuitively, common rationales tend to correspond to the discriminative patterns from the key parts of foreground objects. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective without using any pre-trained object parts or saliency detectors, making it seamlessly to be integrated with the existing SSL process. Specifically, we fit the GradCAM with a branch with limited fitting capacity, which allows the branch to capture the common rationales and discard the less common discriminative patterns. At the test stage, the branch generates a set of spatial weights to selectively aggregate features representing an instance. Extensive experimental results on four visual tasks demonstrate that the proposed method can lead to a significant improvement in different evaluation settings.


page 1

page 3

page 11


Distilling Localization for Self-Supervised Representation Learning

For high-level visual recognition, self-supervised learning defines and ...

Fine-Grained Object Classification via Self-Supervised Pose Alignment

Semantic patterns of fine-grained objects are determined by subtle appea...

Self-Supervised Learning for Fine-Grained Visual Categorization

Recent research in self-supervised learning (SSL) has shown its capabili...

Detail Reinforcement Diffusion Model: Augmentation Fine-Grained Visual Categorization in Few-Shot Conditions

The challenge in fine-grained visual categorization lies in how to explo...

Fine-Grained Self-Supervised Learning with Jigsaw Puzzles for Medical Image Classification

Classifying fine-grained lesions is challenging due to minor and subtle ...

On the Eigenvalues of Global Covariance Pooling for Fine-grained Visual Recognition

The Fine-Grained Visual Categorization (FGVC) is challenging because the...

Please sign up or login with your details

Forgot password? Click here to reset