Improved Analysis of UCRL2 with Empirical Bernstein Inequality

07/10/2020

∙

We consider the problem of exploration-exploitation in communicating Markov Decision Processes. We provide an analysis of UCRL2 with Empirical Bernstein inequalities (UCRL2B). For any MDP with S states, A actions, Γ≤ S next states and diameter D, the regret of UCRL2B is bounded as O(√(DΓ S A T)).

READ FULL TEXT

Improved Analysis of UCRL2 with Empirical Bernstein Inequality

Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes

Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes

Successor Uncertainties: exploration and uncertainty in temporal difference learning

Of Cores: A Partial-Exploration Framework for Markov Decision Processes

Satisfiability Bounds for ω-Regular Properties in Bounded-Parameter Markov Decision Processes

Regret Bounds for Opportunistic Channel Access

Online Allocation and Pricing: Constant Regret via Bellman Inequalities

Improved Analysis of UCRL2 with Empirical Bernstein Inequality

Related Research

Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes

Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes

Successor Uncertainties: exploration and uncertainty in temporal difference learning

Of Cores: A Partial-Exploration Framework for Markov Decision Processes

Satisfiability Bounds for ω-Regular Properties in Bounded-Parameter Markov Decision Processes

Regret Bounds for Opportunistic Channel Access

Online Allocation and Pricing: Constant Regret via Bellman Inequalities