DeepAI AI Chat
Log In Sign Up

Explaining Language Models' Predictions with High-Impact Concepts

by   Ruochen Zhao, et al.

The emergence of large-scale pretrained language models has posed unprecedented challenges in deriving explanations of why the model has made some predictions. Stemmed from the compositional nature of languages, spurious correlations have further undermined the trustworthiness of NLP systems, leading to unreliable model explanations that are merely correlated with the output predictions. To encourage fairness and transparency, there exists an urgent demand for reliable explanations that allow users to consistently understand the model's behavior. In this work, we propose a complete framework for extending concept-based interpretability methods to NLP. Specifically, we propose a post-hoc interpretability method for extracting predictive high-level features (concepts) from the pretrained model's hidden layer activations. We optimize for features whose existence causes the output predictions to change substantially, generates a high impact. Moreover, we devise several evaluation metrics that can be universally applied. Extensive experiments on real and synthetic tasks demonstrate that our method achieves superior results on predictive impact, usability, and faithfulness compared to the baselines.


page 1

page 2

page 3

page 4


Concept-Based Explanations for Tabular Data

The interpretability of machine learning models has been an essential ar...

Explanation Regeneration via Information Bottleneck

Explaining the black-box predictions of NLP models naturally and accurat...

Analyzing the Interpretability Robustness of Self-Explaining Models

Recently, interpretable models called self-explaining models (SEMs) have...

Model Explanations under Calibration

Explaining and interpreting the decisions of recommender systems are bec...

Concept-based Explanations using Non-negative Concept Activation Vectors and Decision Tree for CNN Models

This paper evaluates whether training a decision tree based on concepts ...

CCGen: Explainable Complementary Concept Generation in E-Commerce

We propose and study Complementary Concept Generation (CCGen): given a c...