A Note on the Inception Score

01/06/2018
by   Shane Barratt, et al.
0

Deep generative models are powerful tools that have produced impressive results in recent years. These advances have been for the most part empirically driven, making it essential that we use high quality evaluation metrics. In this paper, we provide new insights into the Inception Score, a recently proposed and widely used evaluation metric for generative models, and demonstrate that it fails to provide useful guidance when comparing models. We discuss both suboptimalities of the metric itself and issues with its application. Finally, we call for researchers to be more systematic and careful when evaluating and comparing generative models, as the advancement of the field depends upon it.

READ FULL TEXT

page 7

page 11

research
06/22/2022

A Study on the Evaluation of Generative Models

Implicit generative models, which do not return likelihood values, such ...
research
11/02/2020

Toward a Generalization Metric for Deep Generative Models

Measuring the generalization capacity of Deep Generative Models (DGMs) i...
research
06/02/2021

Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions

Graph generative models are a highly active branch of machine learning. ...
research
11/19/2022

Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models

In this work, we present some recommendations on the evaluation of state...
research
01/31/2022

On the Robustness of Quality Measures for GANs

This work evaluates the robustness of quality measures of generative mod...
research
12/03/2018

Towards Accurate Generative Models of Video: A New Metric & Challenges

Recent advances in deep generative models have lead to remarkable progre...
research
10/22/2019

Establishing an Evaluation Metric to Quantify Climate Change Image Realism

With success on controlled tasks, generative models are being increasing...

Please sign up or login with your details

Forgot password? Click here to reset