Quantile Markov Decision Process

11/15/2017

∙

In this paper, we consider the problem of optimizing the quantiles of the cumulative rewards of Markov Decision Processes (MDP), to which we refers as Quantile Markov Decision Processes (QMDP). Traditionally, the goal of a Markov Decision Process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly to be infinite). In many applications, however, a decision maker may be interested in optimizing a specific quantile of the cumulative reward instead of its expectation. (If we have some reference here, it would be good.) Our framework of QMDP provides analytical results characterizing the optimal QMDP solution and presents the algorithm for solving the QMDP. We provide analytical results characterizing the optimal QMDP solution and present the algorithms for solving the QMDP. We illustrate the model with two experiments: a grid game and a HIV optimal treatment experiment.

READ FULL TEXT

Quantile Markov Decision Process

Optimizing Quantiles in Preference-based Markov Decision Processes

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization

Optimal Nudging: Solving Average-Reward Semi-Markov Decision Processes as a Minimal Sequence of Cumulative Tasks

Learning Risk Preferences in Markov Decision Processes: an Application to the Fourth Down Decision in Football

Solving Multi-Objective MDP with Lexicographic Preference: An application to stochastic planning with multiple quantile objective

Approximation of Lorenz-Optimal Solutions in Multiobjective Markov Decision Processes

Markov Decision Process for MOOC users behavioral inference

Quantile Markov Decision Process

Related Research

Optimizing Quantiles in Preference-based Markov Decision Processes

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization

Optimal Nudging: Solving Average-Reward Semi-Markov Decision Processes as a Minimal Sequence of Cumulative Tasks

Learning Risk Preferences in Markov Decision Processes: an Application to the Fourth Down Decision in Football

Solving Multi-Objective MDP with Lexicographic Preference: An application to stochastic planning with multiple quantile objective

Approximation of Lorenz-Optimal Solutions in Multiobjective Markov Decision Processes

Markov Decision Process for MOOC users behavioral inference