BERTSurv: BERT-Based Survival Models for Predicting Outcomes of Trauma Patients

by   Yun Zhao, et al.

Survival analysis is a technique to predict the times of specific outcomes, and is widely used in predicting the outcomes for intensive care unit (ICU) trauma patients. Recently, deep learning models have drawn increasing attention in healthcare. However, there is a lack of deep learning methods that can model the relationship between measurements, clinical notes and mortality outcomes. In this paper we introduce BERTSurv, a deep learning survival framework which applies Bidirectional Encoder Representations from Transformers (BERT) as a language representation model on unstructured clinical notes, for mortality prediction and survival analysis. We also incorporate clinical measurements in BERTSurv. With binary cross-entropy (BCE) loss, BERTSurv can predict mortality as a binary outcome (mortality prediction). With partial log-likelihood (PLL) loss, BERTSurv predicts the probability of mortality as a time-to-event outcome (survival analysis). We apply BERTSurv on Medical Information Mart for Intensive Care III (MIMIC III) trauma patient data. For mortality prediction, BERTSurv obtained an area under the curve of receiver operating characteristic curve (AUC-ROC) of 0.86, which is an improvement of 3.6 multilayer perceptron (MLP) without notes. For survival analysis, BERTSurv achieved a concordance index (C-index) of 0.7. In addition, visualizations of BERT's attention heads help to extract patterns in clinical notes and improve model interpretability by showing how the model assigns weights to different inputs.


page 8

page 9

page 13


Early Prediction of Mortality in Critical Care Setting in Sepsis Patients Using Structured Features and Unstructured Clinical Notes

Sepsis is an important cause of mortality, especially in intensive care ...

Leveraging Deep Representations of Radiology Reports in Survival Analysis for Predicting Heart Failure Patient Mortality

Utilizing clinical texts in survival analysis is difficult because they ...

ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes

Accurate mortality prediction allows Intensive Care Units (ICUs) to adeq...

ISeeU: Visually interpretable deep learning for mortality prediction inside the ICU

To improve the performance of Intensive Care Units (ICUs), the field of ...

Using Clinical Drug Representations for Improving Mortality and Length of Stay Predictions

Drug representations have played an important role in cheminformatics. H...

Clinical Utility of the Automatic Phenotype Annotation in Unstructured Clinical Notes: ICU Use Cases

Clinical notes contain information not present elsewhere, including drug...

DeepJoint: Robust Survival Modelling Under Clinical Presence Shift

Observational data in medicine arise as a result of the complex interact...

Please sign up or login with your details

Forgot password? Click here to reset