Zero-Shot Learning via Category-Specific Visual-Semantic Mapping

by   Li Niu, et al.

Zero-Shot Learning (ZSL) aims to classify a test instance from an unseen category based on the training instances from seen categories, in which the gap between seen categories and unseen categories is generally bridged via visual-semantic mapping between the low-level visual feature space and the intermediate semantic space. However, the visual-semantic mapping learnt based on seen categories may not generalize well to unseen categories because the data distributions between seen categories and unseen categories are considerably different, which is known as the projection domain shift problem in ZSL. To address this domain shift issue, we propose a method named Adaptive Embedding ZSL (AEZSL) to learn an adaptive visual-semantic mapping for each unseen category based on the similarities between each unseen category and all the seen categories. Then, we further make two extensions based on our AEZSL method. Firstly, in order to utilize the unlabeled test instances from unseen categories, we extend our AEZSL to a semi-supervised approach named AEZSL with Label Refinement (AEZSL_LR), in which a progressive approach is developed to update the visual classifiers and refine the predicted test labels alternatively based on the similarities among test instances and among unseen categories. Secondly, to avoid learning visual-semantic mapping for each unseen category in the large-scale classification task, we extend our AEZSL to a deep adaptive embedding model named Deep AEZSL (DAEZSL) sharing the similar idea (i.e., visual-semantic mapping should be category-specific and related to the semantic space) with AEZSL, which only needs to be trained once, but can be applied to arbitrary number of unseen categories. Extensive experiments demonstrate that our proposed methods achieve the state-of-the-art results for image classification on four benchmark datasets.


page 9

page 10


Meta-Transfer Networks for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims at recognizing unseen categories using som...

Task-Independent Knowledge Makes for Transferable Representations for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) targets recognizing new categories...

Transductive Zero-Shot Learning with Adaptive Structural Embedding

Zero-shot learning (ZSL) endows the computer vision system with the infe...

Toward Auto-evaluation with Confidence-based Category Relation-aware Regression

Auto-evaluation aims to automatically evaluate a trained model on any te...

Webly Supervised Learning with Category-level Semantic Information

As tons of photos are being uploaded to public websites (e.g., Flickr, B...

Generating CCG Categories

Previous CCG supertaggers usually predict categories using multi-class c...

Zero-Shot Learning with Generative Latent Prototype Model

Zero-shot learning, which studies the problem of object classification f...

Please sign up or login with your details

Forgot password? Click here to reset