Won't you see my neighbor?: User predictions, mental models, and similarity-based explanations of AI classifiers

by   Kimberly Glasgow, et al.

Humans should be able work more effectively with artificial intelligence-based systems when they can predict likely failures and form useful mental models of how the systems work. We conducted a study of human's mental models of artificial intelligence systems using a high-performing image classifier, focusing on participants' ability to predict the classification result for a particular image. Participants viewed individual labeled images in one of two classes and then tried to predict whether the classifier would label them correctly. In this experiment we explored the effect of giving participants additional information about an image's nearest neighbors in a space representing the otherwise uninterpretable features extracted by the lower layers of the classifier's neural network. We found that providing this information did increase participants' prediction performance, and that the performance improvement could be related to the neighbor images' similarity to the target image. We also found indications that the presentation of this information may influence people's own classification of the target image – that is, rather than just anthropomorphizing the system, in some cases the humans become "mechanomorphized" in their judgements.


Visual correspondence-based explanations improve AI robustness and human-AI team accuracy

Explaining artificial intelligence (AI) predictions is increasingly impo...

AdvisingNets: Learning to Distinguish Correct and Wrong Classifications via Nearest-Neighbor Explanations

Besides providing insights into how an image classifier makes its predic...

PERFEX: Classifier Performance Explanations for Trustworthy AI Systems

Explainability of a classification model is crucial when deployed in rea...

A critical reappraisal of predicting suicidal ideation using fMRI

For many psychiatric disorders, neuroimaging offers a potential for revo...

Sequential Explanations with Mental Model-Based Policies

The act of explaining across two parties is a feedback loop, where one p...

Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Program

The advances in artificial intelligence enabled by deep learning archite...

Please sign up or login with your details

Forgot password? Click here to reset