Large Scale Qualitative Evaluation of Generative Image Model Outputs

by   Yannick Assogba, et al.

Evaluating generative image models remains a difficult problem. This is due to the high dimensionality of the outputs, the challenging task of representing but not replicating training data, and the lack of metrics that fully correspond to human perception and capture all the properties we want these models to exhibit. Therefore, qualitative evaluation of model outputs is an important part of model development and research publication practice. Quantitative evaluation is currently under-served by existing tools, which do not easily facilitate structured exploration of a large number of examples across the latent space of the model. To address this issue, we present Ravel, a visual analytics system that enables qualitative evaluation of model outputs on the order of hundreds of thousands of images. Ravel allows users to discover phenomena such as mode collapse, and find areas of training data that the model has failed to capture. It allows users to evaluate both quality and diversity of generated images in comparison to real images or to the output of another model that serves as a baseline. Our paper describes three case studies demonstrating the key insights made possible with Ravel, supported by a domain expert user study.


page 4

page 5

page 6

page 7

page 8

page 9


Smoothing the Generative Latent Space with Mixup-based Distance Learning

Producing diverse and realistic images with generative models such as GA...

Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

In this paper, we present DendroMap, a novel approach to interactively e...

On the Evaluation of Generative Adversarial Networks By Discriminative Models

Generative Adversarial Networks (GANs) can accurately model complex mult...

A Mixed-Initiative Visual Analytics Approach for Qualitative Causal Modeling

Modeling complex systems is a time-consuming, difficult and fragmented t...

DRIT++: Diverse Image-to-Image Translation via Disentangled Representations

Image-to-image translation aims to learn the mapping between two visual ...

AniFaceDrawing: Anime Portrait Exploration during Your Sketching

In this paper, we focus on how artificial intelligence (AI) can be used ...

Siamese neural networks for a generalized, quantitative comparison of complex model outputs

Computational models are quantitative representations of systems. By ana...

Please sign up or login with your details

Forgot password? Click here to reset