Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis

by   Chantal Pellegrini, et al.

Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of annotated data, which often is scarce in the medical domain. Zero-shot methods address this challenge by allowing a flexible adaption to new settings with different clinical findings without relying on labeled data. Further, to integrate automated diagnosis in the clinical workflow, methods should be transparent and explainable, increasing medical professionals' trust and facilitating correctness verification. In this work, we introduce Xplainer, a novel framework for explainable zero-shot diagnosis in the clinical setting. Xplainer adapts the classification-by-description approach of contrastive vision-language models to the multi-label medical diagnosis task. Specifically, instead of directly predicting a diagnosis, we prompt the model to classify the existence of descriptive observations, which a radiologist would look for on an X-Ray scan, and use the descriptor probabilities to estimate the likelihood of a diagnosis. Our model is explainable by design, as the final diagnosis prediction is directly based on the prediction of the underlying descriptors. We evaluate Xplainer on two chest X-ray datasets, CheXpert and ChestX-ray14, and demonstrate its effectiveness in improving the performance and explainability of zero-shot diagnosis. Our results suggest that Xplainer provides a more detailed understanding of the decision-making process and can be a valuable tool for clinical diagnosis.


A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis

Zero-shot medical image classification is a critical process in real-wor...

Knowledge-enhanced Pre-training for Auto-diagnosis of Chest Radiology Images

Despite of the success of multi-modal foundation models pre-trained on l...

Zero-Shot and Few-Shot Learning for Lung Cancer Multi-Label Classification using Vision Transformer

Lung cancer is the leading cause of cancer-related death worldwide. Lung...

ChatGPT-HealthPrompt. Harnessing the Power of XAI in Prompt-Based Healthcare Decision Support using ChatGPT

This study presents an innovative approach to the application of large l...

An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports

The pandemic resulted in vast repositories of unstructured data, includi...

A Simple and Effective Framework for Strict Zero-Shot Hierarchical Classification

In recent years, large language models (LLMs) have achieved strong perfo...

K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging

In this paper, we consider the problem of disease diagnosis. Unlike the ...

Please sign up or login with your details

Forgot password? Click here to reset