Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches

by   Kacper Sokol, et al.

Explanations in Machine Learning come in many forms, but a consensus regarding their desired properties is yet to emerge. In this paper we introduce a taxonomy and a set of descriptors that can be used to characterise and systematically assess explainable systems along five key dimensions: functional, operational, usability, safety and validation. In order to design a comprehensive and representative taxonomy and associated descriptors we surveyed the eXplainable Artificial Intelligence literature, extracting the criteria and desiderata that other authors have proposed or implicitly used in their research. The survey includes papers introducing new explainability algorithms to see what criteria are used to guide their development and how these algorithms are evaluated, as well as papers proposing such criteria from both computer science and social science perspectives. This novel framework allows to systematically compare and contrast explainability approaches, not just to better understand their capabilities but also to identify discrepancies between their theoretical qualities and properties of their implementations. We developed an operationalisation of the framework in the form of Explainability Fact Sheets, which enable researchers and practitioners alike to quickly grasp capabilities and limitations of a particular explainable method. When used as a Work Sheet, our taxonomy can guide the development of new explainability approaches by aiding in their critical evaluation along the five proposed dimensions.


page 13

page 18


Explainable Machine Learning with Prior Knowledge: An Overview

This survey presents an overview of integrating prior knowledge into mac...

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

The rising popularity of explainable artificial intelligence (XAI) to un...

Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications

There has been a growing interest in model-agnostic methods that can mak...

A Survey of Explainable AI and Proposal for a Discipline of Explanation Engineering

In this survey paper, we deep dive into the field of Explainable Artific...

Towards Practical Explainability with Cluster Descriptors

With the rapid development of machine learning, improving its explainabi...

How to Evaluate Explainability? – A Case for Three Criteria

The increasing complexity of software systems and the influence of softw...

Monetary Stabilization in Cryptocurrencies - Design Approaches and Open Questions

The price volatility of cryptocurrencies is often cited as a major hindr...

Please sign up or login with your details

Forgot password? Click here to reset