Predicate correlation learning for scene graph generation

07/06/2021
by   Leitian Tao, et al.
0

For a typical Scene Graph Generation (SGG) method, there is often a large gap in the performance of the predicates' head classes and tail classes. This phenomenon is mainly caused by the semantic overlap between different predicates as well as the long-tailed data distribution. In this paper, a Predicate Correlation Learning (PCL) method for SGG is proposed to address the above two problems by taking the correlation between predicates into consideration. To describe the semantic overlap between strong-correlated predicate classes, a Predicate Correlation Matrix (PCM) is defined to quantify the relationship between predicate pairs, which is dynamically updated to remove the matrix's long-tailed bias. In addition, PCM is integrated into a Predicate Correlation Loss function (L_PC) to reduce discouraging gradients of unannotated classes. The proposed method is evaluated on Visual Genome benchmark, where the performance of the tail classes is significantly improved when built on the existing methods.

READ FULL TEXT

page 1

page 8

research
09/02/2020

PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

Today, scene graph generation(SGG) task is largely limited in realistic ...
research
05/08/2023

LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition

Long-tailed multi-label visual recognition (LTML) task is a highly chall...
research
12/14/2021

Margin Calibration for Long-Tailed Visual Recognition

The long-tailed class distribution in visual recognition tasks poses gre...
research
10/05/2020

Long-tailed Recognition by Routing Diverse Distribution-Aware Experts

Natural data are often long-tail distributed over semantic classes. Exis...
research
05/27/2022

A Survey on Long-Tailed Visual Recognition

The heavy reliance on data is one of the major reasons that currently li...
research
02/07/2023

Delving Deep into Simplicity Bias for Long-Tailed Image Recognition

Simplicity Bias (SB) is a phenomenon that deep neural networks tend to r...
research
06/23/2022

Learning To Generate Scene Graph from Head to Tail

Scene Graph Generation (SGG) represents objects and their interactions w...

Please sign up or login with your details

Forgot password? Click here to reset