Multi-label Few-shot ICD Coding as Autoregressive Generation with Prompt

by   Zhichao Yang, et al.
University of Massachusetts Amherst

Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with an average of 3,000+ tokens. This task is challenging due to the high-dimensional space of multi-label assignment (155,000+ ICD code candidates) and the long-tail challenge - Many ICD codes are infrequently assigned yet infrequent ICD codes are important clinically. This study addresses the long-tail challenge by transforming this multi-label classification task into an autoregressive generation task. Specifically, we first introduce a novel pretraining objective to generate free text diagnoses and procedure using the SOAP structure, the medical logic physicians use for note documentation. Second, instead of directly predicting the high dimensional space of ICD codes, our model generates the lower dimension of text descriptions, which then infer ICD codes. Third, we designed a novel prompt template for multi-label classification. We evaluate our Generation with Prompt model with the benchmark of all code assignment (MIMIC-III-full) and few shot ICD code assignment evaluation benchmark (MIMIC-III-few). Experiments on MIMIC-III-few show that our model performs with a marco F1 30.2, which substantially outperforms the previous MIMIC-III-full SOTA model (marco F1 4.3) and the model specifically designed for few/zero shot setting (marco F1 18.7). Finally, we design a novel ensemble learner, a cross attention reranker with prompts, to integrate previous SOTA and our best few-shot coding predictions. Experiments on MIMIC-III-full show that our ensemble learner substantially improves both macro and micro F1, from 10.4 to 14.6 and from 58.2 to 59.1, respectively.


page 1

page 2

page 3

page 4


Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding

Automatic International Classification of Diseases (ICD) coding aims to ...

Generalized Zero-shot ICD Coding

The International Classification of Diseases (ICD) is a list of classifi...

Improving Predictions of Tail-end Labels using Concatenated BioMed-Transformers for Long Medical Documents

Multi-label learning predicts a subset of labels from a given label set ...

MDACE: MIMIC Documents Annotated with Code Evidence

We introduce a dataset for evidence/rationale extraction on an extreme m...

Medical code prediction with multi-view convolution and description-regularized label-dependent attention

A ubiquitous task in processing electronic medical data is the assignmen...

Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models

Background: Encouraged by the success of pretrained Transformer models i...

Multimodal Machine Learning for Automated ICD Coding

This study presents a multimodal machine learning model to predict ICD-1...

Please sign up or login with your details

Forgot password? Click here to reset