Multi-label Zero-shot Classification by Learning to Transfer from External Knowledge

by   He Huang, et al.

Multi-label zero-shot classification aims to predict multiple unseen class labels for an input image. It is more challenging than its single-label counterpart. On one hand, the unconstrained number of labels assigned to each image makes the model more easily overfit to those seen classes. On the other hand, there is a large semantic gap between seen and unseen classes in the existing multi-label classification datasets. To address these difficult issues, this paper introduces a novel multi-label zero-shot classification framework by learning to transfer from external knowledge. We observe that ImageNet is commonly used to pretrain the feature extractor and has a large and fine-grained label space. This motivates us to exploit it as external knowledge to bridge the seen and unseen classes and promote generalization. Specifically, we construct a knowledge graph including not only classes from the target dataset but also those from ImageNet. Since ImageNet labels are not available in the target dataset, we propose a novel PosVAE module to infer their initial states in the extended knowledge graph. Then we design a relational graph convolutional network (RGCN) to propagate information among classes and achieve knowledge transfer. Experimental results on two benchmark datasets demonstrate the effectiveness of the proposed approach.


page 1

page 2


Multi-Label Zero-Shot Learning with Transfer-Aware Label Embedding Projection

Zero-shot learning transfers knowledge from seen classes to novel unseen...

Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs

Few/Zero-shot learning is a big challenge of many classifications tasks,...

Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation

Zero-Shot Learning (ZSL), which aims at automatically recognizing unseen...

KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification

Recently, Zero-Shot Node Classification (ZNC) has been an emerging and c...

GBE-MLZSL: A Group Bi-Enhancement Framework for Multi-Label Zero-Shot Learning

This paper investigates a challenging problem of zero-shot learning in t...

Semantic Enhanced Knowledge Graph for Large-Scale Zero-Shot Learning

Zero-Shot Learning has been a highlighted research topic in both vision ...

A Relational-learning Perspective to Multi-label Chest X-ray Classification

Multi-label classification of chest X-ray images is frequently performed...

Please sign up or login with your details

Forgot password? Click here to reset