Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling

by   Susobhan Ghosh, et al.

There is a growing interest in using reinforcement learning (RL) to personalize sequences of treatments in digital health to support users in adopting healthier behaviors. Such sequential decision-making problems involve decisions about when to treat and how to treat based on the user's context (e.g., prior activity level, location, etc.). Online RL is a promising data-driven approach for this problem as it learns based on each user's historical responses and uses that knowledge to personalize these decisions. However, to decide whether the RL algorithm should be included in an “optimized” intervention for real-world deployment, we must assess the data evidence indicating that the RL algorithm is actually personalizing the treatments to its users. Due to the stochasticity in the RL algorithm, one may get a false impression that it is learning in certain states and using this learning to provide specific treatments. We use a working definition of personalization and introduce a resampling-based methodology for investigating whether the personalization exhibited by the RL algorithm is an artifact of the RL algorithm stochasticity. We illustrate our methodology with a case study by analyzing the data from a physical activity clinical trial called HeartSteps, which included the use of an online RL algorithm. We demonstrate how our approach enhances data-driven truth-in-advertising of algorithm personalization both across all users as well as within specific users in the study.


page 27

page 28


Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity

With the recent evolution of mobile health technologies, health scientis...

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines

Online reinforcement learning (RL) algorithms are increasingly used to p...

Effective Warm Start for the Online Actor-Critic Reinforcement Learning based mHealth Intervention

Online reinforcement learning (RL) is increasingly popular for the perso...

Hindsight Learning for MDPs with Exogenous Inputs

We develop a reinforcement learning (RL) framework for applications that...

A User Study on Explainable Online Reinforcement Learning for Adaptive Systems

Online reinforcement learning (RL) is increasingly used for realizing ad...

Discovering User Types: Mapping User Traits by Task-Specific Behaviors in Reinforcement Learning

When assisting human users in reinforcement learning (RL), we can repres...

Reinforcement Learning for Multi-Objective Optimization of Online Decisions in High-Dimensional Systems

This paper describes a purely data-driven solution to a class of sequent...

Please sign up or login with your details

Forgot password? Click here to reset