Encoding High-Level Visual Attributes in Capsules for Explainable Medical Diagnoses

by   Rodney LaLonde, et al.

Deep neural networks are often called black-boxes due to their difficult-to-interpret decisions. This is characteristic of a deeper trend in machine learning, where predictive performance typically comes at the cost of interpretability. In some domains, such as image-based diagnostic tasks, understanding the reasons behind machine generated predictions is vital in assessing trust. In this study, we introduce novel designs of capsule networks to provide explainable diagnoses. Our proposed deep explainable capsule architecture, called DX-Caps, can encode high-level visual attributes within the vectors of capsules in order to simultaneously produce malignancy predictions for lung cancer as well as approximations of six visually-interpretable attributes, used by radiologists to explain their predictions. To reduce parameter and memory burden of this deeper network, we introduce a new capsule-average pooling function. With this simple, but fundamental addition, capsule networks can be designed in a deeper fashion than was possible before. Our overall approach can be characterized as multi-task learning; we learn to approximate the six high-level visual attributes of a lung nodule within the vectors of our uniquely constructed deep capsule network, while simultaneously segmenting the nodule and predicting its malignancy potential (diagnosis). Tested on over 1000 CT scans, our experimental results show that our proposed algorithm can approximate the visual attributes of lung nodules far better than a deep multi-path dense 3D CNN. The proposed network also achieves higher diagnostic accuracy than a baseline explainable capsule network X-Caps and CapsNet when applied to this task for the first time as well. To the best of our knowledge, this is the first study to investigate capsule networks for visual attribute prediction in general, and explainable medical image diagnosis in particular.


page 2

page 7


MIXCAPS: A Capsule Network-based Mixture of Experts for Lung Nodule Malignancy Prediction

Lung diseases including infections such as Pneumonia, Tuberculosis, and ...

An Interpretable Deep Hierarchical Semantic Convolutional Neural Network for Lung Nodule Malignancy Classification

While deep learning methods are increasingly being applied to tasks such...

DeepCaps: Going Deeper with Capsule Networks

Capsule Network is a promising concept in deep learning, yet its true po...

Joint Learning for Pulmonary Nodule Segmentation, Attributes and Malignancy Prediction

Refer to the literature of lung nodule classification, many studies adop...

Attention-Enhanced Cross-Task Network for Analysing Multiple Attributes of Lung Nodules in CT

Accurate characterisation of visual attributes such as spiculation, lobu...

Learning More for Free - A Multi Task Learning Approach for Improved Pathology Classification in Capsule Endoscopy

The progress in Computer Aided Diagnosis (CADx) of Wireless Capsule Endo...

Please sign up or login with your details

Forgot password? Click here to reset