Hierarchical Label-wise Attention Transformer Model for Explainable ICD Coding

by   Leibo Liu, et al.

International Classification of Diseases (ICD) coding plays an important role in systematically classifying morbidity and mortality data. In this study, we propose a hierarchical label-wise attention Transformer model (HiLAT) for the explainable prediction of ICD codes from clinical documents. HiLAT firstly fine-tunes a pretrained Transformer model to represent the tokens of clinical documents. We subsequently employ a two-level hierarchical label-wise attention mechanism that creates label-specific document representations. These representations are in turn used by a feed-forward neural network to predict whether a specific ICD code is assigned to the input clinical document of interest. We evaluate HiLAT using hospital discharge summaries and their corresponding ICD-9 codes from the MIMIC-III database. To investigate the performance of different types of Transformer models, we develop ClinicalplusXLNet, which conducts continual pretraining from XLNet-Base using all the MIMIC-III clinical notes. The experiment results show that the F1 scores of the HiLAT+ClinicalplusXLNet outperform the previous state-of-the-art models for the top-50 most frequent ICD-9 codes from MIMIC-III. Visualisations of attention weights present a potential explainability tool for checking the face validity of ICD code predictions.


page 1

page 2

page 3

page 4


Medical Code Prediction from Discharge Summary: Document to Sequence BERT using Sequence Attention

Clinical notes are unstructured text generated by clinicians during pati...

Description-based Label Attention Classifier for Explainable ICD-9 Classification

ICD-9 coding is a relevant clinical billing task, where unstructured tex...

Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation

Diagnostic or procedural coding of clinical notes aims to derive a coded...

An Explainable CNN Approach for Medical Codes Prediction from Clinical Text

Method: We develop CNN-based methods for automatic ICD coding based on c...

Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset

Clinical coding is currently a labour-intensive, error-prone, but critic...

Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models

Background: Encouraged by the success of pretrained Transformer models i...

ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network

Automated ICD coding, which assigns the International Classification of ...

Please sign up or login with your details

Forgot password? Click here to reset