Benchmarking and Analyzing Generative Data for Visual Recognition

07/25/2023
by   Bo Li, et al.
0

Advancements in large pre-trained generative models have expanded their potential as effective data generators in visual recognition. This work delves into the impact of generative images, primarily comparing paradigms that harness external data (generative retrieval original). Our key contributions are: 1) GenBench Construction: We devise GenBench, a broad benchmark comprising 22 datasets with 2548 categories, to appraise generative data across various visual recognition tasks. 2) CLER Score: To address the insufficient correlation of existing metrics (, FID, CLIP score) with downstream recognition performance, we propose CLER, a training-free metric indicating generative data's efficiency for recognition tasks prior to training. 3) New Baselines: Comparisons of generative data with retrieved data from the same external pool help to elucidate the unique traits of generative data. 4) External Knowledge Injection: By fine-tuning special token embeddings for each category via Textual Inversion, performance improves across 17 datasets, except when dealing with low-resolution reference images. Our exhaustive benchmark and analysis spotlight generative data's promise in visual recognition, while identifying key challenges for future investigation.

READ FULL TEXT

page 2

page 8

page 12

page 14

page 15

page 16

04/03/2023

Vision-Language Models for Vision Tasks: A Survey

Most visual recognition studies rely heavily on crowd-labelled data in d...
09/12/2023

Dynamic Visual Prompt Tuning for Parameter Efficient Transfer Learning

Parameter efficient transfer learning (PETL) is an emerging research spo...
09/18/2023

Parameter-Efficient Long-Tailed Recognition

The "pre-training and fine-tuning" paradigm in addressing long-tailed re...
05/20/2021

Opening Deep Neural Networks with Generative Models

Image classification methods are usually trained to perform predictions ...
05/28/2023

Plug-and-Play Knowledge Injection for Pre-trained Language Models

Injecting external knowledge can improve the performance of pre-trained ...

Please sign up or login with your details

Forgot password? Click here to reset