Generalized Category Discovery

by   Sagar Vaze, et al.

In this paper, we consider a highly general image recognition setting wherein, given a labelled and unlabelled set of images, the task is to categorize all images in the unlabelled set. Here, the unlabelled images may come from labelled classes or from novel ones. Existing recognition methods are not able to deal with this setting, because they make several restrictive assumptions, such as the unlabelled instances only coming from known - or unknown - classes and the number of unknown classes being known a-priori. We address the more unconstrained setting, naming it 'Generalized Category Discovery', and challenge all these assumptions. We first establish strong baselines by taking state-of-the-art algorithms from novel category discovery and adapting them for this task. Next, we propose the use of vision transformers with contrastive representation learning for this open world setting. We then introduce a simple yet effective semi-supervised k-means method to cluster the unlabelled data into seen and unseen classes automatically, substantially outperforming the baselines. Finally, we also propose a new approach to estimate the number of classes in the unlabelled data. We thoroughly evaluate our approach on public datasets for generic object classification including CIFAR10, CIFAR100 and ImageNet-100, and for fine-grained visual recognition including CUB, Stanford Cars and Herbarium19, benchmarking on this new setting to foster future research.


page 1

page 8

page 13


CiPR: An Efficient Framework with Cross-instance Positive Relations for Generalized Category Discovery

We tackle the issue of generalized category discovery (GCD). GCD conside...

OpenGCD: Assisting Open World Recognition with Generalized Category Discovery

A desirable open world recognition (OWR) system requires performing thre...

XCon: Learning with Experts for Fine-grained Category Discovery

We address the problem of generalized category discovery (GCD) in this p...

PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery

Although existing semi-supervised learning models achieve remarkable suc...

AutoNovel: Automatically Discovering and Learning Novel Visual Categories

We tackle the problem of discovering novel classes in an image collectio...

MetaGCD: Learning to Continually Learn in Generalized Category Discovery

In this paper, we consider a real-world scenario where a model that is t...

Learning to Discover Novel Visual Categories via Deep Transfer Clustering

We consider the problem of discovering novel object categories in an ima...

Please sign up or login with your details

Forgot password? Click here to reset