Predicting the Need for Blood Transfusion in Intensive Care Units with Reinforcement Learning

by   Yuqing Wang, et al.

As critically ill patients frequently develop anemia or coagulopathy, transfusion of blood products is a frequent intervention in the Intensive Care Units (ICU). However, inappropriate transfusion decisions made by physicians are often associated with increased risk of complications and higher hospital costs. In this work, we aim to develop a decision support tool that uses available patient information for transfusion decision-making on three common blood products (red blood cells, platelets, and fresh frozen plasma). To this end, we adopt an off-policy batch reinforcement learning (RL) algorithm, namely, discretized Batch Constrained Q-learning, to determine the best action (transfusion or not) given observed patient trajectories. Simultaneously, we consider different state representation approaches and reward design mechanisms to evaluate their impacts on policy learning. Experiments are conducted on two real-world critical care datasets: the MIMIC-III and the UCSF. Results demonstrate that policy recommendations on transfusion achieved comparable matching against true hospital policies via accuracy and weighted importance sampling evaluations on the MIMIC-III dataset. Furthermore, a combination of transfer learning (TL) and RL on the data-scarce UCSF dataset can provide up to 17.02 improvement in jump-start and asymptotic performance in terms of weighted importance sampling averaged over three transfusion tasks. Finally, simulations on transfusion decisions suggest that the transferred RL policy could reduce patients' estimated 28-day mortality rate by 2.74 1.18


page 5

page 7


A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units

The management of invasive mechanical ventilation, and the regulation of...

Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation

Our aim is to establish a framework where reinforcement learning (RL) of...

Reinforcement Learning Assisted Oxygen Therapy for COVID-19 Patients Under Intensive Care

Patients with severe Coronavirus disease 19 (COVID-19) typically require...

Blood lactate concentration prediction in critical care patients: handling missing values

Blood lactate concentration is a strong indicator of mortality risk in c...

An Optimal Policy for Patient Laboratory Tests in Intensive Care Units

Laboratory testing is an integral tool in the management of patient care...

Prediction of Blood Lactate Values in Critically Ill Patients: A Retrospective Multi-center Cohort Study

Purpose. Elevations in initially obtained serum lactate levels are stron...

Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment

Many batch RL health applications first discretize time into fixed inter...

Please sign up or login with your details

Forgot password? Click here to reset