Generating Counterfactual Explanations with Natural Language

by   Lisa Anne Hendricks, et al.

Natural language explanations of deep neural network decisions provide an intuitive way for a AI agent to articulate a reasoning process. Current textual explanations learn to discuss class discriminative features in an image. However, it is also helpful to understand which attributes might change a classification decision if present in an image (e.g., "This is not a Scarlet Tanager because it does not have black wings.") We call such textual explanations counterfactual explanations, and propose an intuitive method to generate counterfactual explanations by inspecting which evidence in an input is missing, but might contribute to a different classification decision if present in the image. To demonstrate our method we consider a fine-grained image classification task in which we take as input an image and a counterfactual class and output text which explains why the image does not belong to a counterfactual class. We then analyze our generated counterfactual explanations both qualitatively and quantitatively using proposed automatic metrics.


Counterfactual Visual Explanations

A counterfactual query is typically of the form 'For situation X, why wa...

Faithfulness Tests for Natural Language Explanations

Explanations of neural models aim to reveal a model's decision-making pr...

DeDUCE: Generating Counterfactual Explanations Efficiently

When an image classifier outputs a wrong class label, it can be helpful ...

Counterfactual Explanations for Natural Language Interfaces

A key challenge facing natural language interfaces is enabling users to ...

Counterfactual Explanations for Concepts in ℰℒℋ

Knowledge bases are widely used for information management on the web, e...

Anytime Generation of Counterfactual Explanations for Text Classification

In many machine learning applications, it is important for the user to u...

Generating Visual Explanations

Clearly explaining a rationale for a classification decision to an end-u...

Please sign up or login with your details

Forgot password? Click here to reset